[PATCH 00/59] Deduplicating CTF linker

[PATCH 00/59] Deduplicating CTF linker

[Nick Alcock]

So the deduplicating CTF linker is finally ready for review.  (Note: ready
for *review*, probably not yet quite ready for committing, since to speed
things up a bit I've posted before all non-Linux-arch tests are finished.
In particular, big-endian tests, Solaris tests, and tests on mingw and
cygwin will be done in parallel with this review.  Cross build-and-checks to
every supported arch, --enable-targets=all tests, and 64-bit and 32-bit
tests on x86-64 Linux and FreeBSD have been done already.)

Please read and review as you wish -- most of these are libctf-only changes,
but I'd be very happy for review of those too.  There are only a few dozen
lines of non-testsuite changes outside libctf.

Before the actual linker changes comes a lot of loosely-related stuff in
libctf.

First, there's new infrastructure, mostly used by the deduplicating linker
but also to improve interaction with the linker itself and other consumers,
like GDB:
 - proper textual error/warning reporting (adopted by all in-binutils
   consumers): libctf errors are no longer just an errno value and an error
   message swallowed up in the debug stream
 - adding new error codes only needs to be done in one place, and produces
   fewer relocs in the final binary
 - new API functions to get things we were concealing before, like the
   number of members in an enum, or to do things previously impossible,
   like bump the refcount on a ctf_file_t so you can hold it open for
   longer, and a bunch of new API functions for the dynhash
 - a new dynhash-like thing, the dynset (doesn't store a value, but is
   much more memory-efficient), still based on the libiberty hashtab
 - a whole new type of iterator, the *_next iterator family, much easier
   to use than the existing *_iter iterators that take a callback: I hope
   to move away from use of the old *_iter functions in most cases within
   libctf, and eventually reimplement *_iter in terms of *_next

Then there are a lot of bugfixes, and performance and memory-usage improvements,
again mostly triggered by the needs of the deduplicator.

Once those are past we have linker-related changes, starting on patch 43,
including one big commit (patch 50) containing the CTF deduplicator itself.
(I can cut this one up if you find it too big, but control roughly flows
from the top of the patch to the bottom anyway, so slicing it into pieces
doesn't really buy much in terms of readability.)

Nearly all the changes are in libctf: ld gets only a couple of new options
to control various new CTF linker settings and a better way of reporting
(arbitrarily many) libctf errors and warnings.  No changes are needed in
libbfd at all.

Most of the new work is in the new file libctf/ctf-dedup.c, which
deduplicates the types section of CTF dicts, taking a bunch of CTF dicts and
linking them together into a single output dict and possibly a set of child
dicts containing conflicting types and (link mode allowing) types that
appear in only one input dict.  This gets done by recursively hashing types
(terminating the recursion at tagged structures and unions so as not to get
caught in cycles), identifying types with ambiguous definitions (multiple
hash values for the same name), then emitting one type per deduplicated type
hash value into appropriate output dicts.  The algorithm used by ctf-dedup.c
is documented at the top of the file.

The changes in ctf-link.c tie that deduplicator into the rest of libctf, fix
bugs in the existing variable-section linking code, allow the variable
section to be omitted from the output, allow CTF dicts to be opened lazily
by libctf itself rather than requiring callers to do it, start using the new
err/warning infrastructure, and fix a bunch of bugs.

There is a new configure option which turns on debug output during the
initial type hashing phase: this generates so very much output that it
massively slows ld down even when LIBCTF_DEBUG is not set in the
environment.  So it's turned off by default.  When that configure flag is
not provided, CTF-related operations are not at all the slowest thing ld
does, even though the deduplicator is still serial (though deduplication can
still take a few seconds for bigger projects): see below for some example
timings, some of which are approximate because the timings are frankly in
the noise.  Deduplication *is* a bit memory-hungry, mostly due to hashtab
overhead, but again this is tolerable and will be reduced further in time.

ld gains new options to customize the link mode (share-unconflicted or
share-duplicated: see ld.texi for the meaning of this) and to emit the
variable section if desired.  (Normal links will not want a variable
section, because the things in there correspond to static variables that
have no symbol table entry and cannot be looked up in any useful fashion.)

There is also a new CTF ld testsuite, largely due to Egeyar Bagcioglu: it
tests linking of CTF sections (including conflicting type names in different
TUs landing in suitable child dicts) and operation on a bunch of invalid
inputs: binutils gets some formatting fixes and an enhancement to
run_dump_test to let it compile things before assembling them.
Unfortunately because the pseudos GCC generates to emit assembler output are
often nonportable, the testsuite works by compiling C into asm and then
assembling and linking that: so a compiler capable of CTF generation via -gt
is needed to run the new tests.  We hope to fix this in time, but it will
take GCC changes.

Finally there is an important bugfix: our trick to ensure that the output
has only one .ctf section broke sometime in the last six months or so of
upstream development, so we were getting outputs with huge numbers of empty
CTF sections in: the new approach should be more robust.


I am reasonably confident in the algorithm, though no doubt bugs will
emerge.  CTF file format changes should not require significant changes in
the deduplicator, because we try wherever possible to use the public libctf
API in the dedup code rather than digging into the guts of the file format.
Much of this is probably reusable when CTF is extended to handle other
languages, as well (e.g. we don't really depend on any particular detail of
the C type system to break cycles, only that there *is* some thing or set of
things which must be present in order for a cycle in the type graph to
exist).


Coverage analysis suggests that the tests now in the testsuite cover all of
ctf-dedup.c except for paths associated with ld -r (input dicts with
children), paths associated with types that CTF cannot represent, a couple
of lines associated with cu-mapped links, and paths associated with the
share-duplicated link mode: perhaps 80% of the total line count is covered.
More tests will be added later to cover the missing cases.


Some speed (cpu-user-time) and CTF size figures are below: they were taken
on my 2.4GHz x86-64 Linux workhorse box.  The GCC used was a GCC 10 snapshot
from mid-Feb plus CTF-generation patches: see
https://github.com/oracle/gcc.git oracle/ctf-gen (if you want to replicate
this but don't want to fetch from GitHub I have a git bundle from upstream
commit 7a775242ea296849a34ce27de179eaaec411e880 available at
<http://www.esperi.org.uk/~nix/bundles/gcc-ctf.bundle>).

We start with easy stuff and proceed to things with crazy numbers of types
or that do mad things in the build process as a bit of a regression test,
notes below.  Sorry this is over 80 chars wide: I hope it's readable anyway:

Program        DWARF size .ctf size                                         Time[3]
               [1]        Compressed Uncompressed              Largest  -gt    no -gt
                                                  (no strings) input TU
GNU ld         1150314    42479      146945       63623        142226   17.33  16.76
libbfd         2746704    71380      237445       129929       143970   43.25  41.61
ld (static)    4490908    86428      289947       170933       162226   59.72  58.0 
gas            1063842    48537      161929       75643        148271   17.66  16.66
coreutils ls   164517     6455       19384        15673        18481    0.30   0.33
vim            4740693    111080     372398       281471       138455   176.21 175.46
libX11         9525236    45799      155109       125534       58964    300.11 298.40
emacs 26.3 [2] 8079794    86267      303306       225828       174808   938.52 933.78
GhostScript    18972151   2311683[4] 4828538      2726368      127543   265.73 252.41
Gtk 3.24.20    11895771   255101[5]  908719       717034       255101   1268   1245
CLISP[6]       2870363    49234      193162       124811       254536   160.90 151.42
X.org xserver  6718404    105955     369395       295470       257878   448    436

[1] .debug_info only: perhaps I should include .debug_str?  Really what we
    need is "type only" DWARF for comparison, but that requires compiler
    hacking I haven't looked at yet.
[2] times include all lisp compilation
[3] times do not include configury: a good few of them are in the noise.
    This at least shows that the deduplicator won't slow things down too
    badly.
[4] GhostScript is a worst case for huge numbers of types with identical
    names but distinct definitions in different TUs.  Format v4 gains the
    ability to share the definitions of such types between TUs, and will
    reduce this size back down to something more like the size of the main
    dict, 113983 bytes.  (The deduplicator will hardly need to change: the
    changes are elsewhere in libctf.)
[5] Slightly afflicted by the same things as GhostScript: the CTF size will
    probably halve in format v4.
[6] size of lisp.a's sole member: clisp is an oddball, and would probably
    require a specialized reader that knew something about the .mem dumping
    process.

Egeyar Bagcioglu (3):
  libctf, types: ensure the emission of ECTF_NOPARENT
  ld: Reformat CTF errors into warnings.
  ld: new CTF testsuite

Nick Alcock (56):
  include, libctf: typo fixes
  libctf: restructure error handling to reduce relocations
  libctf, create: support addition of references to the unimplemented
    type
  libctf, create: do not corrupt function types' arglists at insertion
    time
  libctf, create: add explicit casts for variables' and slices' types
  libctf, types: allow ctf_type_reference of dynamic slices
  libctf, open: drop unnecessary historical wart around forwards
  libctf, create: member names of "" and NULL should be the same
  libctf, create: fix addition of anonymous struct/union members
  libctf, create: empty dicts are dirty to start with
  libctf, types: support slices of anything terminating in an int
  libctf, types: ints, floats and typedefs with no name are invalid
  libctf, archive: stop ctf_arc_bufopen triggering crazy unmaps
  libctf: having debugging enabled is unlikely
  libctf: add ctf_type_name_raw
  libctf: add ctf_type_kind_forwarded
  libctf: add ctf_member_count
  libctf: add ctf_archive_count
  libctf: fix __extension__ with non-GNU C compilers
  libctf: add new dynhash functions
  libctf, hash: improve insertion of existing keys into dynhashes
  libctf, hash: save per-item space when no key/item freeing function
  libctf, hash: introduce the ctf_dynset
  libctf: move existing inlines into ctf-inlines.h
  libctf: add ctf_forwardable_kind
  libctf: add ctf_ref
  libctf, next: introduce new class of easier-to-use iterators
  libctf, next, hash: add dynhash and dynset _next iteration
  libctf: pass the thunk down properly when wrapping qsort_r
  libctf: error out on corrupt CTF with invalid header flags
  libctf, ld, binutils: add textual error/warning reporting for libctf
  libctf, types: enhance ctf_type_aname to print function arg types
  libctf, decl: avoid leaks of the formatted string on error
  libctf, dump: migrate towards dumping errors rather than truncation
  libctf, dump: fix slice dumping
  libctf, open: fix opening CTF in binaries with no symtab
  libctf, archive: fix bad error message
  libctf: check for vasprintf
  libctf: rename the type_mapping_key to type_key
  libctf: sort out potential refcount loops
  libctf: drop error-prone ctf_strerror
  libctf, link: add lazy linking: clean up input members: err/warn
    cleanup
  libctf, link: fix ctf_link_write fd leak
  libctf, link: redo cu-mapping handling
  ctf, link: fix spurious conflicts of variables in the variable section
  libctf, link: add the ability to filter out variables from the link
  libctf: add SHA-1 support for libctf
  libctf, dedup: add new configure option --enable-libctf-hash-debugging
  libctf, dedup: add deduplicator
  libctf, link: add CTF_LINK_OMIT_VARIABLES_SECTION
  libctf, link: tie in the deduplicating linker
  binutils: objdump: ctf: drop incorrect linefeeds
  ld: new options --ctf-variables and --ctf-share-types
  binutils, testsuite: allow compilation before doing run_dump_test
  ld, testsuite: only run CTF tests when ld and GCC support CTF
  ld: do not produce one empty output .ctf section for every input .ctf

 binutils/objdump.c                            |   25 +-
 binutils/readelf.c                            |   23 +-
 binutils/testsuite/lib/binutils-common.exp    |   58 +-
 include/ctf-api.h                             |  178 +-
 include/ctf.h                                 |    3 +-
 ld/Makefile.am                                |    7 +-
 ld/Makefile.in                                |    8 +-
 ld/NEWS                                       |   10 +
 ld/configure                                  |    6 +-
 ld/configure.ac                               |    1 +
 ld/ld.h                                       |    8 +
 ld/ld.texi                                    |   34 +
 ld/ldlang.c                                   |   63 +-
 ld/ldlex.h                                    |    3 +
 ld/lexsup.c                                   |   29 +
 ld/testsuite/ld-ctf/A-2.c                     |    6 +
 ld/testsuite/ld-ctf/A.c                       |    5 +
 ld/testsuite/ld-ctf/B-2.c                     |    5 +
 ld/testsuite/ld-ctf/B.c                       |    4 +
 ld/testsuite/ld-ctf/C-2.c                     |    5 +
 ld/testsuite/ld-ctf/C.c                       |    5 +
 ld/testsuite/ld-ctf/array-char.c              |    2 +
 ld/testsuite/ld-ctf/array-int.c               |    1 +
 ld/testsuite/ld-ctf/array.d                   |   34 +
 ld/testsuite/ld-ctf/child-float.c             |    4 +
 ld/testsuite/ld-ctf/child-int.c               |    4 +
 ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d |   40 +
 ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d |   41 +
 .../ld-ctf/conflicting-cycle-1.parent.d       |   38 +
 ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d |   40 +
 ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d |   41 +
 .../ld-ctf/conflicting-cycle-2.parent.d       |   40 +
 ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d |   39 +
 ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d |   40 +
 .../ld-ctf/conflicting-cycle-3.parent.d       |   37 +
 ld/testsuite/ld-ctf/conflicting-enums.d       |   35 +
 ld/testsuite/ld-ctf/conflicting-typedefs.d    |   33 +
 ld/testsuite/ld-ctf/cross-tu-1.c              |   12 +
 ld/testsuite/ld-ctf/cross-tu-2.c              |    8 +
 ld/testsuite/ld-ctf/cross-tu-conflicting-2.c  |    8 +
 ld/testsuite/ld-ctf/cross-tu-cyclic-1.c       |   14 +
 ld/testsuite/ld-ctf/cross-tu-cyclic-2.c       |   16 +
 ld/testsuite/ld-ctf/cross-tu-cyclic-3.c       |    3 +
 ld/testsuite/ld-ctf/cross-tu-cyclic-4.c       |    4 +
 .../ld-ctf/cross-tu-cyclic-conflicting.d      |   57 +
 .../ld-ctf/cross-tu-cyclic-nonconflicting.d   |   50 +
 ld/testsuite/ld-ctf/cross-tu-into-cycle.d     |   64 +
 ld/testsuite/ld-ctf/cross-tu-noncyclic.d      |   46 +
 ld/testsuite/ld-ctf/ctf.exp                   |   31 +
 ld/testsuite/ld-ctf/cycle-1.c                 |    7 +
 ld/testsuite/ld-ctf/cycle-1.d                 |   36 +
 ld/testsuite/ld-ctf/cycle-2.A.d               |   40 +
 ld/testsuite/ld-ctf/cycle-2.B.d               |   40 +
 ld/testsuite/ld-ctf/cycle-2.C.d               |   40 +
 ld/testsuite/ld-ctf/diag-ctf-version-0.d      |    5 +
 ld/testsuite/ld-ctf/diag-ctf-version-0.s      |   44 +
 .../diag-ctf-version-2-unsupported-feature.d  |    5 +
 .../diag-ctf-version-2-unsupported-feature.s  |   44 +
 ld/testsuite/ld-ctf/diag-ctf-version-f.d      |    5 +
 ld/testsuite/ld-ctf/diag-ctf-version-f.s      |   44 +
 ld/testsuite/ld-ctf/diag-cttname-invalid.d    |    5 +
 ld/testsuite/ld-ctf/diag-cttname-invalid.s    |   44 +
 ld/testsuite/ld-ctf/diag-cttname-null.d       |   24 +
 ld/testsuite/ld-ctf/diag-cttname-null.s       |   44 +
 ld/testsuite/ld-ctf/diag-cuname.d             |   39 +
 ld/testsuite/ld-ctf/diag-cuname.s             |   44 +
 .../ld-ctf/diag-decompression-failure.d       |    5 +
 .../ld-ctf/diag-decompression-failure.s       |   44 +
 ld/testsuite/ld-ctf/diag-parlabel.d           |   39 +
 ld/testsuite/ld-ctf/diag-parlabel.s           |   44 +
 ld/testsuite/ld-ctf/diag-parname.d            |    5 +
 ld/testsuite/ld-ctf/diag-parname.s            |   44 +
 ld/testsuite/ld-ctf/diag-unsupported-flag.d   |    5 +
 ld/testsuite/ld-ctf/diag-unsupported-flag.s   |   44 +
 .../ld-ctf/diag-wrong-magic-number-mixed.d    |   39 +
 ld/testsuite/ld-ctf/diag-wrong-magic-number.d |    5 +
 ld/testsuite/ld-ctf/diag-wrong-magic-number.s |   44 +
 ld/testsuite/ld-ctf/enum-2.c                  |    3 +
 ld/testsuite/ld-ctf/enum.c                    |    3 +
 ld/testsuite/ld-ctf/function.c                |    3 +
 ld/testsuite/ld-ctf/function.d                |   23 +
 ld/testsuite/ld-ctf/slice.c                   |    6 +
 ld/testsuite/ld-ctf/slice.d                   |   30 +
 ld/testsuite/ld-ctf/super-sub-cycles.c        |   10 +
 ld/testsuite/ld-ctf/super-sub-cycles.d        |   34 +
 ld/testsuite/ld-ctf/typedef-int.c             |    3 +
 ld/testsuite/ld-ctf/typedef-long.c            |    3 +
 ld/testsuite/ld-ctf/union-1.c                 |    4 +
 ld/testsuite/lib/ld-lib.exp                   |   52 +
 libctf/.gitignore                             |    1 +
 libctf/Makefile.am                            |   12 +-
 libctf/Makefile.in                            |  360 +-
 libctf/aclocal.m4                             |    1 +
 libctf/config.h.in                            |    7 +
 libctf/configure                              |   73 +-
 libctf/configure.ac                           |    8 +-
 libctf/ctf-archive.c                          |  147 +-
 libctf/ctf-create.c                           |  112 +-
 libctf/ctf-decl.c                             |    5 +-
 libctf/ctf-decls.h                            |    2 +-
 libctf/ctf-dedup.c                            | 3157 +++++++++++++++++
 libctf/ctf-dump.c                             |  199 +-
 libctf/ctf-error.c                            |  102 +-
 libctf/ctf-hash.c                             |  560 ++-
 libctf/ctf-impl.h                             |  305 +-
 libctf/ctf-inlines.h                          |   97 +
 libctf/ctf-link.c                             | 1191 ++++++-
 libctf/ctf-lookup.c                           |   19 +-
 libctf/ctf-open-bfd.c                         |   85 +-
 libctf/ctf-open.c                             |   91 +-
 libctf/ctf-sha1.c                             |   50 +
 libctf/ctf-sha1.h                             |   41 +
 libctf/ctf-subr.c                             |  101 +-
 libctf/ctf-types.c                            |  508 ++-
 libctf/ctf-util.c                             |   66 +
 libctf/libctf.ver                             |   18 +-
 libctf/mkerrors.sed                           |   28 +
 117 files changed, 8997 insertions(+), 619 deletions(-)
 create mode 100644 ld/testsuite/ld-ctf/A-2.c
 create mode 100644 ld/testsuite/ld-ctf/A.c
 create mode 100644 ld/testsuite/ld-ctf/B-2.c
 create mode 100644 ld/testsuite/ld-ctf/B.c
 create mode 100644 ld/testsuite/ld-ctf/C-2.c
 create mode 100644 ld/testsuite/ld-ctf/C.c
 create mode 100644 ld/testsuite/ld-ctf/array-char.c
 create mode 100644 ld/testsuite/ld-ctf/array-int.c
 create mode 100644 ld/testsuite/ld-ctf/array.d
 create mode 100644 ld/testsuite/ld-ctf/child-float.c
 create mode 100644 ld/testsuite/ld-ctf/child-int.c
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-enums.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-typedefs.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-1.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-conflicting-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-1.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-3.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-4.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-conflicting.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-into-cycle.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-noncyclic.d
 create mode 100644 ld/testsuite/ld-ctf/ctf.exp
 create mode 100644 ld/testsuite/ld-ctf/cycle-1.c
 create mode 100644 ld/testsuite/ld-ctf/cycle-1.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.A.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.B.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.C.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-0.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-0.s
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-f.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-f.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-invalid.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-invalid.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-null.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-null.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cuname.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cuname.s
 create mode 100644 ld/testsuite/ld-ctf/diag-decompression-failure.d
 create mode 100644 ld/testsuite/ld-ctf/diag-decompression-failure.s
 create mode 100644 ld/testsuite/ld-ctf/diag-parlabel.d
 create mode 100644 ld/testsuite/ld-ctf/diag-parlabel.s
 create mode 100644 ld/testsuite/ld-ctf/diag-parname.d
 create mode 100644 ld/testsuite/ld-ctf/diag-parname.s
 create mode 100644 ld/testsuite/ld-ctf/diag-unsupported-flag.d
 create mode 100644 ld/testsuite/ld-ctf/diag-unsupported-flag.s
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number-mixed.d
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number.d
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number.s
 create mode 100644 ld/testsuite/ld-ctf/enum-2.c
 create mode 100644 ld/testsuite/ld-ctf/enum.c
 create mode 100644 ld/testsuite/ld-ctf/function.c
 create mode 100644 ld/testsuite/ld-ctf/function.d
 create mode 100644 ld/testsuite/ld-ctf/slice.c
 create mode 100644 ld/testsuite/ld-ctf/slice.d
 create mode 100644 ld/testsuite/ld-ctf/super-sub-cycles.c
 create mode 100644 ld/testsuite/ld-ctf/super-sub-cycles.d
 create mode 100644 ld/testsuite/ld-ctf/typedef-int.c
 create mode 100644 ld/testsuite/ld-ctf/typedef-long.c
 create mode 100644 ld/testsuite/ld-ctf/union-1.c
 create mode 100644 libctf/.gitignore
 create mode 100644 libctf/ctf-dedup.c
 create mode 100644 libctf/ctf-inlines.h
 create mode 100644 libctf/ctf-sha1.c
 create mode 100644 libctf/ctf-sha1.h
 create mode 100644 libctf/mkerrors.sed

-- 
2.27.0.247.g3dff7de930

[PATCH 01/59] include, libctf: typo fixes

[Nick Alcock]

include/
	* ctf-api.h: Fix typos in comments.
libctf/
	* ctf-impl.h: Fix typos in comments.
---
 include/ctf-api.h | 6 +++---
 libctf/ctf-impl.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index d6b05bc71fe..bb1cf0f50eb 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -30,7 +30,7 @@
 
 #ifdef	__cplusplus
 extern "C"
-  {
+{
 #endif
 
 /* Clients can open one or more CTF containers and obtain a pointer to an
@@ -80,7 +80,7 @@ typedef struct ctf_link_sym
 
 /* Indication of how to share types when linking.  */
 
-/* Share all types thare are not in conflict.  The default.  */
+/* Share all types that are not in conflict.  The default.  */
 #define CTF_LINK_SHARE_UNCONFLICTED 0x0
 
 /* Share only types that are used by multiple inputs.  Not implemented yet.  */
@@ -428,7 +428,7 @@ extern unsigned char *ctf_link_write (ctf_file_t *, size_t *size,
 				      size_t threshold);
 
 /* Specialist linker functions.  These functions are not used by ld, but can be
-   used by other prgorams making use of the linker machinery for other purposes
+   used by other programs making use of the linker machinery for other purposes
    to customize its output.  */
 extern int ctf_link_add_cu_mapping (ctf_file_t *, const char *from,
 				    const char *to);
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 133ca0ee6e2..b20a4f05a80 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -37,7 +37,7 @@
 
 #ifdef	__cplusplus
 extern "C"
-  {
+{
 #endif
 
 /* Compiler attributes.  */
-- 
2.27.0.247.g3dff7de930

[PATCH 02/59] libctf: restructure error handling to reduce relocations

[Nick Alcock]

Jose Marchesi noted that the traditional-Unix error array in ctf-error.c
introduces one reloc per error to initialize the array: 58 so far.  We
can reduce this to zero using an array of carefully-sized individual
members which is used to construct a string table, that is then
referenced by the lookup functions: but doing this automatically is a
pain.

Bruno Haible wrote suitable code years ago: I got permission to reuse it
(Bruno says "... which I hereby put in the public domain"); I modified
it a tiny bit (similarly to what Ulrich Drepper did in the dsohowto
text, but I redid it from scratch), commented it up a bit, and shifted
the error table into that form, migrating it into the new file
ctf-error.h.

This has the advantage that it spotted both typos in the text of the
errors in the comments in ctf-api.h and typos in the error defines in
the comments in ctf-error.c, and places where the two were simply not
in sync.  All are now fixed.

One new constant exists in ctf-api.h: CTF_NERR, since the old method of
working out the number of errors in ctf-error.c was no longer usable,
and it seems that the number of CTF errors is something users might
reasonably want as well.  It should be pretty easy to keep up to date as
new errors are introduced.

include/
	* ctf-api.h (ECTF_*): Improve comments.
	(ECTF_NERR): New.

libctf/
	* ctf-error.c: Include <stddef.h>, for offsetof.
	(_ctf_errlist): Migrate to...
	(_ctf_errlist_t): ... this.
	(_ctf_erridx): New, indexes into _ctf_errlist_t.
	(_ctf_nerr): Remove.
	(ctf_errmsg): Adjust accordingly.
	* Makefile.am (BUILT_SOURCES): Note...
	(ctf-error.h): ... this new rule.
	* Makefile.in: Regenerate.
	* mkerrors.sed: New, process ctf-api.h to generate ctf-error.h.
	* .gitignore: New, ignore ctf-error.h.
---
 include/ctf-api.h   | 106 ++++++++++++++++++++++----------------------
 libctf/.gitignore   |   1 +
 libctf/Makefile.am  |   5 +++
 libctf/Makefile.in  |  15 +++++--
 libctf/ctf-error.c  |  99 +++++++++++++++++------------------------
 libctf/mkerrors.sed |  28 ++++++++++++
 6 files changed, 140 insertions(+), 114 deletions(-)
 create mode 100644 libctf/.gitignore
 create mode 100644 libctf/mkerrors.sed

diff --git a/include/ctf-api.h b/include/ctf-api.h
index bb1cf0f50eb..2e3e28b840c 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -153,60 +153,62 @@ typedef struct ctf_snapshot_id
 
 enum
   {
-   ECTF_FMT = ECTF_BASE,	/* File is not in CTF or ELF format.  */
-   ECTF_BFDERR,			/* BFD error.  */
-   ECTF_CTFVERS,		/* CTF version is more recent than libctf.  */
-   ECTF_BFD_AMBIGUOUS,		/* Ambiguous BFD target.  */
-   ECTF_SYMTAB,			/* Symbol table uses invalid entry size.  */
-   ECTF_SYMBAD,			/* Symbol table data buffer invalid.  */
-   ECTF_STRBAD,			/* String table data buffer invalid.  */
-   ECTF_CORRUPT,		/* File data corruption detected.  */
-   ECTF_NOCTFDATA,		/* ELF file does not contain CTF data.  */
-   ECTF_NOCTFBUF,		/* Buffer does not contain CTF data.  */
-   ECTF_NOSYMTAB,		/* Symbol table data is not available.  */
-   ECTF_NOPARENT,		/* Parent CTF container is not available.  */
-   ECTF_DMODEL,			/* Data model mismatch.  */
-   ECTF_LINKADDEDLATE,		/* File added to link too late.  */
-   ECTF_ZALLOC,			/* Failed to allocate (de)compression buffer.  */
-   ECTF_DECOMPRESS,		/* Failed to decompress CTF data.  */
-   ECTF_STRTAB,			/* String table for this string is missing.  */
-   ECTF_BADNAME,		/* String offset is corrupt w.r.t. strtab.  */
-   ECTF_BADID,			/* Invalid type ID number.  */
-   ECTF_NOTSOU,			/* Type is not a struct or union.  */
-   ECTF_NOTENUM,		/* Type is not an enum.  */
-   ECTF_NOTSUE,			/* Type is not a struct, union, or enum.  */
-   ECTF_NOTINTFP,		/* Type is not an integer, float, or enum.  */
-   ECTF_NOTARRAY,		/* Type is not an array.  */
-   ECTF_NOTREF,			/* Type does not reference another type.  */
-   ECTF_NAMELEN,		/* Buffer is too small to hold type name.  */
-   ECTF_NOTYPE,			/* No type found corresponding to name.  */
-   ECTF_SYNTAX,			/* Syntax error in type name.  */
-   ECTF_NOTFUNC,		/* Symbol entry or type is not a function.  */
-   ECTF_NOFUNCDAT,		/* No func info available for function.  */
-   ECTF_NOTDATA,		/* Symtab entry does not refer to a data obj.  */
-   ECTF_NOTYPEDAT,		/* No type info available for object.  */
-   ECTF_NOLABEL,		/* No label found corresponding to name.  */
-   ECTF_NOLABELDATA,		/* File does not contain any labels.  */
-   ECTF_NOTSUP,			/* Feature not supported.  */
-   ECTF_NOENUMNAM,		/* Enum element name not found.  */
-   ECTF_NOMEMBNAM,		/* Member name not found.  */
-   ECTF_RDONLY,			/* CTF container is read-only.  */
-   ECTF_DTFULL,			/* CTF type is full (no more members allowed).  */
-   ECTF_FULL,			/* CTF container is full.  */
-   ECTF_DUPLICATE,		/* Duplicate member or variable name.  */
-   ECTF_CONFLICT,		/* Conflicting type definition present.  */
-   ECTF_OVERROLLBACK,		/* Attempt to roll back past a ctf_update.  */
-   ECTF_COMPRESS,		/* Failed to compress CTF data.  */
-   ECTF_ARCREATE,		/* Error creating CTF archive.  */
-   ECTF_ARNNAME,		/* Name not found in CTF archive.  */
-   ECTF_SLICEOVERFLOW,		/* Overflow of type bitness or offset in slice.  */
-   ECTF_DUMPSECTUNKNOWN,	/* Unknown section number in dump.  */
-   ECTF_DUMPSECTCHANGED,	/* Section changed in middle of dump.  */
-   ECTF_NOTYET,			/* Feature not yet implemented.  */
-   ECTF_INTERNAL,		/* Internal error in link.  */
-   ECTF_NONREPRESENTABLE	/* Type not representable in CTF.  */
+   ECTF_FMT = ECTF_BASE, /* File is not in CTF or ELF format.  */
+   ECTF_BFDERR,		/* BFD error.  */
+   ECTF_CTFVERS,	/* CTF dict version is too new for libctf.  */
+   ECTF_BFD_AMBIGUOUS,	/* Ambiguous BFD target.  */
+   ECTF_SYMTAB,		/* Symbol table uses invalid entry size.  */
+   ECTF_SYMBAD,		/* Symbol table data buffer is not valid.  */
+   ECTF_STRBAD,		/* String table data buffer is not valid.  */
+   ECTF_CORRUPT,	/* File data structure corruption detected.  */
+   ECTF_NOCTFDATA,	/* File does not contain CTF data.  */
+   ECTF_NOCTFBUF,	/* Buffer does not contain CTF data.  */
+   ECTF_NOSYMTAB,	/* Symbol table information is not available.  */
+   ECTF_NOPARENT,	/* The parent CTF dictionary is unavailable.  */
+   ECTF_DMODEL,		/* Data model mismatch.  */
+   ECTF_LINKADDEDLATE,	/* File added to link too late.  */
+   ECTF_ZALLOC,		/* Failed to allocate (de)compression buffer.  */
+   ECTF_DECOMPRESS,	/* Failed to decompress CTF data.  */
+   ECTF_STRTAB,		/* External string table is not available.  */
+   ECTF_BADNAME,	/* String name offset is corrupt.  */
+   ECTF_BADID,		/* Invalid type identifier.  */
+   ECTF_NOTSOU,		/* Type is not a struct or union.  */
+   ECTF_NOTENUM,	/* Type is not an enum.  */
+   ECTF_NOTSUE,		/* Type is not a struct, union, or enum.  */
+   ECTF_NOTINTFP,	/* Type is not an integer, float, or enum.  */
+   ECTF_NOTARRAY,	/* Type is not an array.  */
+   ECTF_NOTREF,		/* Type does not reference another type.  */
+   ECTF_NAMELEN,	/* Buffer is too small to hold type name.  */
+   ECTF_NOTYPE,		/* No type found corresponding to name.  */
+   ECTF_SYNTAX,		/* Syntax error in type name.  */
+   ECTF_NOTFUNC,	/* Symbol table entry or type is not a function.  */
+   ECTF_NOFUNCDAT,	/* No function information available for function.  */
+   ECTF_NOTDATA,	/* Symbol table entry does not refer to a data object.  */
+   ECTF_NOTYPEDAT,	/* No type information available for symbol.  */
+   ECTF_NOLABEL,	/* No label found corresponding to name.  */
+   ECTF_NOLABELDATA,	/* File does not contain any labels.  */
+   ECTF_NOTSUP,		/* Feature not supported.  */
+   ECTF_NOENUMNAM,	/* Enum element name not found.  */
+   ECTF_NOMEMBNAM,	/* Member name not found.  */
+   ECTF_RDONLY,		/* CTF container is read-only.  */
+   ECTF_DTFULL,		/* CTF type is full (no more members allowed).  */
+   ECTF_FULL,		/* CTF container is full.  */
+   ECTF_DUPLICATE,	/* Duplicate member or variable name.  */
+   ECTF_CONFLICT,	/* Conflicting type is already defined.  */
+   ECTF_OVERROLLBACK,	/* Attempt to roll back past a ctf_update.  */
+   ECTF_COMPRESS,	/* Failed to compress CTF data.  */
+   ECTF_ARCREATE,	/* Error creating CTF archive.  */
+   ECTF_ARNNAME,	/* Name not found in CTF archive.  */
+   ECTF_SLICEOVERFLOW,	/* Overflow of type bitness or offset in slice.  */
+   ECTF_DUMPSECTUNKNOWN, /* Unknown section number in dump.  */
+   ECTF_DUMPSECTCHANGED, /* Section changed in middle of dump.  */
+   ECTF_NOTYET,		/* Feature not yet implemented.  */
+   ECTF_INTERNAL,	/* Internal error in link.  */
+   ECTF_NONREPRESENTABLE /* Type not representable in CTF.  */
   };
 
+#define ECTF_NERR (ECTF_NONREPRESENTABLE - ECTF_BASE + 1)	/* Count of CTF errors.  */
+
 /* The CTF data model is inferred to be the caller's data model or the data
    model of the given object, unless ctf_setmodel() is explicitly called.  */
 #define	CTF_MODEL_ILP32 1	/* Object data model is ILP32.  */
diff --git a/libctf/.gitignore b/libctf/.gitignore
new file mode 100644
index 00000000000..b95386e97d3
--- /dev/null
+++ b/libctf/.gitignore
@@ -0,0 +1 @@
+ctf-error.h
diff --git a/libctf/Makefile.am b/libctf/Makefile.am
index c672d07e142..cdd3a5c55d6 100644
--- a/libctf/Makefile.am
+++ b/libctf/Makefile.am
@@ -52,3 +52,8 @@ libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
 libctf_la_LDFLAGS = $(libctf_nobfd_la_LDFLAGS)
 libctf_la_SOURCES = $(libctf_nobfd_la_SOURCES) ctf-open-bfd.c
+
+BUILT_SOURCES = ctf-error.h
+
+ctf-error.h: $(srcdir)/mkerrors.sed $(srcdir)/../include/ctf-api.h
+	sed -nf $(srcdir)/mkerrors.sed < $(srcdir)/../include/ctf-api.h > $@
diff --git a/libctf/Makefile.in b/libctf/Makefile.in
index 9e6f2551a56..a26fa31da49 100644
--- a/libctf/Makefile.in
+++ b/libctf/Makefile.in
@@ -452,7 +452,8 @@ libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
 libctf_la_LDFLAGS = $(libctf_nobfd_la_LDFLAGS)
 libctf_la_SOURCES = $(libctf_nobfd_la_SOURCES) ctf-open-bfd.c
-all: config.h
+BUILT_SOURCES = ctf-error.h
+all: $(BUILT_SOURCES) config.h
 	$(MAKE) $(AM_MAKEFLAGS) all-am
 
 .SUFFIXES:
@@ -855,13 +856,15 @@ distcleancheck: distclean
 	       $(distcleancheck_listfiles) ; \
 	       exit 1; } >&2
 check-am: all-am
-check: check-am
+check: $(BUILT_SOURCES)
+	$(MAKE) $(AM_MAKEFLAGS) check-am
 all-am: Makefile $(LTLIBRARIES) $(HEADERS) config.h
 installdirs:
 	for dir in "$(DESTDIR)$(libdir)" "$(DESTDIR)$(includedir)"; do \
 	  test -z "$$dir" || $(MKDIR_P) "$$dir"; \
 	done
-install: install-am
+install: $(BUILT_SOURCES)
+	$(MAKE) $(AM_MAKEFLAGS) install-am
 install-exec: install-exec-am
 install-data: install-data-am
 uninstall: uninstall-am
@@ -891,6 +894,7 @@ distclean-generic:
 maintainer-clean-generic:
 	@echo "This command is intended for maintainers to use"
 	@echo "it deletes files that may require special tools to rebuild."
+	-test -z "$(BUILT_SOURCES)" || rm -f $(BUILT_SOURCES)
 clean: clean-am
 
 clean-am: clean-generic clean-libLTLIBRARIES clean-libtool \
@@ -965,7 +969,7 @@ ps-am:
 
 uninstall-am: uninstall-includeHEADERS uninstall-libLTLIBRARIES
 
-.MAKE: all install-am install-strip
+.MAKE: all check install install-am install-strip
 
 .PHONY: CTAGS GTAGS TAGS all all-am am--refresh check check-am clean \
 	clean-cscope clean-generic clean-libLTLIBRARIES clean-libtool \
@@ -989,6 +993,9 @@ uninstall-am: uninstall-includeHEADERS uninstall-libLTLIBRARIES
 .PRECIOUS: Makefile
 
 
+ctf-error.h: $(srcdir)/mkerrors.sed $(srcdir)/../include/ctf-api.h
+	sed -nf $(srcdir)/mkerrors.sed < $(srcdir)/../include/ctf-api.h > $@
+
 # Tell versions [3.59,3.63) of GNU make to not export all variables.
 # Otherwise a system limit (for SysV at least) may be exceeded.
 .NOEXPORT:
diff --git a/libctf/ctf-error.c b/libctf/ctf-error.c
index c457aa0a29f..1e69672007a 100644
--- a/libctf/ctf-error.c
+++ b/libctf/ctf-error.c
@@ -18,71 +18,54 @@
    <http://www.gnu.org/licenses/>.  */
 
 #include <ctf-impl.h>
+#include <stddef.h>
 
-static const char *const _ctf_errlist[] = {
-  "File is not in CTF or ELF format",		     /* ECTF_FMT */
-  "BFD error",					     /* ECTF_BFDERR */
-  "File uses more recent CTF version than libctf",   /* ECTF_CTFVERS */
-  "Ambiguous BFD target",			     /* ECTF_BFD_AMBIGUOUS */
-  "Symbol table uses invalid entry size",	     /* ECTF_SYMTAB */
-  "Symbol table data buffer is not valid",	     /* ECTF_SYMBAD */
-  "String table data buffer is not valid",	     /* ECTF_STRBAD */
-  "File data structure corruption detected",	     /* ECTF_CORRUPT */
-  "File does not contain CTF data",		     /* ECTF_NOCTFDATA */
-  "Buffer does not contain CTF data",		     /* ECTF_NOCTFBUF */
-  "Symbol table information is not available",	     /* ECTF_NOSYMTAB */
-  "Type information is in parent and unavailable",   /* ECTF_NOPARENT */
-  "Cannot import types with different data model",   /* ECTF_DMODEL */
-  "File added to link too late",		     /* ECTF_LINKADDEDLATE */
-  "Failed to allocate (de)compression buffer",	     /* ECTF_ZALLOC */
-  "Failed to decompress CTF data",		     /* ECTF_DECOMPRESS */
-  "External string table is not available",	     /* ECTF_STRTAB */
-  "String name offset is corrupt",		     /* ECTF_BADNAME */
-  "Invalid type identifier",			     /* ECTF_BADID */
-  "Type is not a struct or union",		     /* ECTF_NOTSOU */
-  "Type is not an enum",			     /* ECTF_NOTENUM */
-  "Type is not a struct, union, or enum",	     /* ECTF_NOTSUE */
-  "Type is not an integer, float, or enum",	     /* ECTF_NOTINTFP */
-  "Type is not an array",			     /* ECTF_NOTARRAY */
-  "Type does not reference another type",	     /* ECTF_NOTREF */
-  "Input buffer is too small for type name",	     /* ECTF_NAMELEN */
-  "No type information available for that name",     /* ECTF_NOTYPE */
-  "Syntax error in type name",			     /* ECTF_SYNTAX */
-  "Symbol table entry or type is not a function",    /* ECTF_NOTFUNC */
-  "No function information available for symbol",    /* ECTF_NOFUNCDAT */
-  "Symbol table entry is not a data object",	     /* ECTF_NOTDATA */
-  "No type information available for symbol",	     /* ECTF_NOTYPEDAT */
-  "No label information available for that name",    /* ECTF_NOLABEL */
-  "File does not contain any labels",		     /* ECTF_NOLABELDATA */
-  "Feature not supported",			     /* ECTF_NOTSUP */
-  "Invalid enum element name",			     /* ECTF_NOENUMNAM */
-  "Invalid member name",			     /* ECTF_NOMEMBNAM */
-  "CTF container is read-only",			     /* ECTF_RDONLY */
-  "Limit on number of dynamic type members reached", /* ECTF_DTFULL */
-  "Limit on number of dynamic types reached",	     /* ECTF_FULL */
-  "Duplicate member or variable name",		     /* ECTF_DUPLICATE */
-  "Conflicting type is already defined",	     /* ECTF_CONFLICT */
-  "Attempt to roll back past a ctf_update",	     /* ECTF_OVERROLLBACK */
-  "Failed to compress CTF data",		     /* ECTF_COMPRESS */
-  "Failed to create CTF archive",		     /* ECTF_ARCREATE */
-  "Name not found in CTF archive",		     /* ECTF_ARNNAME */
-  "Overflow of type bitness or offset in slice",     /* ECTF_SLICEOVERFLOW */
-  "Unknown section number in dump",		     /* ECTF_DUMPSECTUNKNOWN */
-  "Section changed in middle of dump",		     /* ECTF_DUMPSECTCHANGED */
-  "Feature not yet implemented",		     /* ECTF_NOTYET */
-  "Internal error in link",			     /* ECTF_INTERNAL */
-  "Type not representable in CTF"		     /* ECTF_NONREPRESENTABLE */
-};
-
-static const int _ctf_nerr = sizeof (_ctf_errlist) / sizeof (_ctf_errlist[0]);
+/* This construct is due to Bruno Haible: much thanks.  */
+
+/* Give each structure member a unique name.  The name does not matter, so we
+   use the line number in ctf-error.h to uniquify them.  */
+
+#define ERRSTRFIELD(line) ERRSTRFIELD1 (line)
+#define ERRSTRFIELD1(line) ctf_errstr##line
+
+/* The error message strings, each in a unique structure member precisely big
+   enough for that error, plus a str member to access them all as a string
+   table.  */
+
+static const union _ctf_errlist_t
+{
+  __extension__ struct
+  {
+#define _S(n, s) char ERRSTRFIELD (__LINE__) [sizeof (s)];
+#include "ctf-error.h"
+#undef _S
+  };
+  char str[1];
+} _ctf_errlist =
+  {
+   {
+#define _S(n, s) s,
+#include "ctf-error.h"
+#undef _S
+   }
+  };
+
+/* Offsets to each member in the string table, computed using offsetof.  */
+
+static const unsigned int _ctf_erridx[] =
+  {
+#define _S(n, s) [n - ECTF_BASE] = offsetof (union _ctf_errlist_t, ERRSTRFIELD (__LINE__)),
+#include "ctf-error.h"
+#undef _S
+  };
 
 const char *
 ctf_errmsg (int error)
 {
   const char *str;
 
-  if (error >= ECTF_BASE && (error - ECTF_BASE) < _ctf_nerr)
-    str = _ctf_errlist[error - ECTF_BASE];
+  if (error >= ECTF_BASE && (error - ECTF_BASE) < ECTF_NERR)
+    str = _ctf_errlist.str + _ctf_erridx[error - ECTF_BASE];
   else
     str = ctf_strerror (error);
 
diff --git a/libctf/mkerrors.sed b/libctf/mkerrors.sed
new file mode 100644
index 00000000000..146fd4abc1d
--- /dev/null
+++ b/libctf/mkerrors.sed
@@ -0,0 +1,28 @@
+#
+#   Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; see the file COPYING.  If not see
+# <http://www.gnu.org/licenses/>.
+#
+
+# Only process lines in the error-define block
+/= ECTF_BASE/,/ECTF_NERR/ {
+    # Do not process non-errors (braces, ECTF_NERR, etc).
+    /^ *ECTF_/!n;
+    # Strip out the base initializer.
+    s, = ECTF_BASE,,;
+    # Transform errors into _S(...).
+    s@^ *\(ECTF_[^[:blank:],]*\),\{0,1\}[[:blank:]]*/\* \(.*\).  \*/$@_S (\1, "\2")@;
+    p;
+  }
-- 
2.27.0.247.g3dff7de930

[PATCH 03/59] libctf, create: support addition of references to the unimplemented type

[Nick Alcock]

The deduplicating linker adds types from the linker inputs to the output
via the same API everyone else does, so it's important that we can emit
everything that the compiler wants us to.  Unfortunately, the compiler
may represent the unimplemented type (used for compiler constructs that
CTF cannot currently encode) as type zero or as a type of kind
CTF_K_UNKNOWN, and we don't allow the addition of types that cite the
former.

Adding this support adds a tiny bit of extra complexity: additions of
structure members immediately following a member of the unimplemented
type must be via ctf_add_member_offset or ctf_add_member_encoded, since
we have no idea how big members of the unimplemented type are.
(Attempts to do otherwise return -ECTF_NONREPRESENTABLE, like other
attempts to do forbidden things with the unimplemented type.)

Even slices of the unimplemented type are permitted: this is the only
case in which you can slice a type that terminates in a non-integral
type, on the grounds that it was likely integral in the source code,
it's just that we can't represent that sort of integral type properly
yet.

libctf/
	* ctf-create.c (ctf_add_reftype): Support refs to type zero.
	(ctf_add_array): Support array contents of type zero.
	(ctf_add_function): Support arguments and return types of
	type zero.
	(ctf_add_typedef): Support typedefs to type zero.
	(ctf_add_member_offset): Support members of type zero,
	unless added at unspecified (naturally-aligned) offset.
---
 libctf/ctf-create.c | 42 ++++++++++++++++++++++++++++++++++--------
 1 file changed, 34 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 808da372de8..67a3f199a96 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -900,7 +900,7 @@ ctf_add_reftype (ctf_file_t *fp, uint32_t flag, ctf_id_t ref, uint32_t kind)
   if (ref == CTF_ERR || ref > CTF_MAX_TYPE)
     return (ctf_set_errno (fp, EINVAL));
 
-  if (ctf_lookup_by_id (&tmp, ref) == NULL)
+  if (ref != 0 && ctf_lookup_by_id (&tmp, ref) == NULL)
     return CTF_ERR;		/* errno is set for us.  */
 
   if ((type = ctf_add_generic (fp, flag, NULL, kind, &dtd)) == CTF_ERR)
@@ -957,12 +957,13 @@ ctf_add_slice (ctf_file_t *fp, uint32_t flag, ctf_id_t ref,
   if (ref == CTF_ERR || ref > CTF_MAX_TYPE)
     return (ctf_set_errno (fp, EINVAL));
 
-  if ((tp = ctf_lookup_by_id (&tmp, ref)) == NULL)
+  if (ref != 0 && ((tp = ctf_lookup_by_id (&tmp, ref)) == NULL))
     return CTF_ERR;		/* errno is set for us.  */
 
   kind = ctf_type_kind_unsliced (tmp, ref);
   if ((kind != CTF_K_INTEGER) && (kind != CTF_K_FLOAT) &&
-      (kind != CTF_K_ENUM))
+      (kind != CTF_K_ENUM)
+      && (ref != 0))
     return (ctf_set_errno (fp, ECTF_NOTINTFP));
 
   if ((type = ctf_add_generic (fp, flag, NULL, CTF_K_SLICE, &dtd)) == CTF_ERR)
@@ -1008,7 +1009,8 @@ ctf_add_array (ctf_file_t *fp, uint32_t flag, const ctf_arinfo_t *arp)
   if (arp == NULL)
     return (ctf_set_errno (fp, EINVAL));
 
-  if (ctf_lookup_by_id (&tmp, arp->ctr_contents) == NULL)
+  if (arp->ctr_contents != 0
+      && ctf_lookup_by_id (&tmp, arp->ctr_contents) == NULL)
     return CTF_ERR;		/* errno is set for us.  */
 
   tmp = fp;
@@ -1062,13 +1064,14 @@ ctf_add_function (ctf_file_t *fp, uint32_t flag,
   if (ctc->ctc_flags & CTF_FUNC_VARARG)
     vlen++;	       /* Add trailing zero to indicate varargs (see below).  */
 
-  if (ctf_lookup_by_id (&tmp, ctc->ctc_return) == NULL)
+  if (ctc->ctc_return != 0
+      && ctf_lookup_by_id (&tmp, ctc->ctc_return) == NULL)
     return CTF_ERR;		/* errno is set for us.  */
 
   for (i = 0; i < ctc->ctc_argc; i++)
     {
       tmp = fp;
-      if (ctf_lookup_by_id (&tmp, argv[i]) == NULL)
+      if (argv[i] != 0 && ctf_lookup_by_id (&tmp, argv[i]) == NULL)
 	return CTF_ERR;		/* errno is set for us.  */
     }
 
@@ -1259,7 +1262,7 @@ ctf_add_typedef (ctf_file_t *fp, uint32_t flag, const char *name,
   if (ref == CTF_ERR || ref > CTF_MAX_TYPE)
     return (ctf_set_errno (fp, EINVAL));
 
-  if (ctf_lookup_by_id (&tmp, ref) == NULL)
+  if (ref != 0 && ctf_lookup_by_id (&tmp, ref) == NULL)
     return CTF_ERR;		/* errno is set for us.  */
 
   if ((type = ctf_add_generic (fp, flag, name, CTF_K_TYPEDEF,
@@ -1387,7 +1390,20 @@ ctf_add_member_offset (ctf_file_t *fp, ctf_id_t souid, const char *name,
 
   if ((msize = ctf_type_size (fp, type)) < 0 ||
       (malign = ctf_type_align (fp, type)) < 0)
-    return -1;			/* errno is set for us.  */
+    {
+      /* The unimplemented type, and any type that resolves to it, has no size
+	 and no alignment: it can correspond to any number of compiler-inserted
+	 types.  */
+
+      if (ctf_errno (fp) == ECTF_NONREPRESENTABLE)
+	{
+	  msize = 0;
+	  malign = 0;
+	  ctf_set_errno (fp, 0);
+	}
+      else
+	return -1;		/* errno is set for us.  */
+    }
 
   if ((dmd = malloc (sizeof (ctf_dmdef_t))) == NULL)
     return (ctf_set_errno (fp, EAGAIN));
@@ -1415,6 +1431,16 @@ ctf_add_member_offset (ctf_file_t *fp, ctf_id_t souid, const char *name,
 	  ctf_encoding_t linfo;
 	  ssize_t lsize;
 
+	  /* Propagate any error from ctf_type_resolve.  If the last member was
+	     of unimplemented type, this may be -ECTF_NONREPRESENTABLE: we
+	     cannot insert right after such a member without explicit offset
+	     specification, because its alignment and size is not known.  */
+	  if (ltype == CTF_ERR)
+	    {
+	      free (dmd);
+	      return -1;	/* errno is set for us.  */
+	    }
+
 	  if (ctf_type_encoding (fp, ltype, &linfo) == 0)
 	    off += linfo.cte_bits;
 	  else if ((lsize = ctf_type_size (fp, ltype)) > 0)
-- 
2.27.0.247.g3dff7de930

[PATCH 04/59] libctf, create: do not corrupt function types' arglists at insertion time

[Nick Alcock]

ctf_add_function assumes that function types' arglists are of type
ctf_id_t.  Since they are CTF IDs, they are 32 bits wide, a uint32_t:
unfortunately ctf_id_t is a forward-compatible user-facing 64 bits wide,
and should never ever reach the CTF storage level.

All the CTF code other than ctf_add_function correctly assumes that
function arglists outside dynamic containers are 32 bits wide, so the
serialization machinery ends up cutting off half the arglist, corrupting
all args but the first (a good sign is a bunch of args of ID 0, the
unimplemented type, popping up).

Fix this by copying the arglist into place item by item, casting it
properly, at the same time as we validate the arg types.  Fix the type
of the dtu_argv in the dynamic container and drop the now-unnecessary
cast in the serializer.

libctf/
	* ctf-impl.h (ctf_dtdef_t) <dtu_argv>: Fix type.
	* ctf-create.c (ctf_add_function): Check for unimplemented type
	and populate at the same time.  Populate one-by-one, not via
	memcpy.
	(ctf_serialize): Remove unnecessary cast.
	* ctf-types.c (ctf_func_type_info): Likewise.
	(ctf_func_type_args): Likewise.  Fix comment typo.
---
 libctf/ctf-create.c | 23 +++++++++++++----------
 libctf/ctf-impl.h   |  2 +-
 libctf/ctf-types.c  |  6 +++---
 3 files changed, 17 insertions(+), 14 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 67a3f199a96..f91da6ea159 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -448,7 +448,7 @@ ctf_serialize (ctf_file_t *fp)
 	    uint32_t argc;
 
 	    for (argc = 0; argc < vlen; argc++)
-	      *argv++ = (uint32_t) dtd->dtd_u.dtu_argv[argc];
+	      *argv++ = dtd->dtd_u.dtu_argv[argc];
 
 	    if (vlen & 1)
 	      *argv++ = 0;	/* Pad to 4-byte boundary.  */
@@ -1052,7 +1052,7 @@ ctf_add_function (ctf_file_t *fp, uint32_t flag,
   ctf_dtdef_t *dtd;
   ctf_id_t type;
   uint32_t vlen;
-  ctf_id_t *vdat = NULL;
+  uint32_t *vdat = NULL;
   ctf_file_t *tmp = fp;
   size_t i;
 
@@ -1068,19 +1068,23 @@ ctf_add_function (ctf_file_t *fp, uint32_t flag,
       && ctf_lookup_by_id (&tmp, ctc->ctc_return) == NULL)
     return CTF_ERR;		/* errno is set for us.  */
 
-  for (i = 0; i < ctc->ctc_argc; i++)
-    {
-      tmp = fp;
-      if (argv[i] != 0 && ctf_lookup_by_id (&tmp, argv[i]) == NULL)
-	return CTF_ERR;		/* errno is set for us.  */
-    }
-
   if (vlen > CTF_MAX_VLEN)
     return (ctf_set_errno (fp, EOVERFLOW));
 
   if (vlen != 0 && (vdat = malloc (sizeof (ctf_id_t) * vlen)) == NULL)
     return (ctf_set_errno (fp, EAGAIN));
 
+  for (i = 0; i < ctc->ctc_argc; i++)
+    {
+      tmp = fp;
+      if (argv[i] != 0 && ctf_lookup_by_id (&tmp, argv[i]) == NULL)
+	{
+	  free (vdat);
+	  return CTF_ERR;	   /* errno is set for us.  */
+	}
+      vdat[i] = (uint32_t) argv[i];
+    }
+
   if ((type = ctf_add_generic (fp, flag, NULL, CTF_K_FUNCTION,
 			       &dtd)) == CTF_ERR)
     {
@@ -1091,7 +1095,6 @@ ctf_add_function (ctf_file_t *fp, uint32_t flag,
   dtd->dtd_data.ctt_info = CTF_TYPE_INFO (CTF_K_FUNCTION, flag, vlen);
   dtd->dtd_data.ctt_type = (uint32_t) ctc->ctc_return;
 
-  memcpy (vdat, argv, sizeof (ctf_id_t) * ctc->ctc_argc);
   if (ctc->ctc_flags & CTF_FUNC_VARARG)
     vdat[vlen - 1] = 0;		   /* Add trailing zero to indicate varargs.  */
   dtd->dtd_u.dtu_argv = vdat;
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index b20a4f05a80..fdd48f0d331 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -172,7 +172,7 @@ typedef struct ctf_dtdef
     ctf_list_t dtu_members;	/* struct, union, or enum */
     ctf_arinfo_t dtu_arr;	/* array */
     ctf_encoding_t dtu_enc;	/* integer or float */
-    ctf_id_t *dtu_argv;		/* function */
+    uint32_t *dtu_argv;		/* function */
     ctf_slice_t dtu_slice;	/* slice */
   } dtd_u;
 } ctf_dtdef_t;
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index cd910434ea4..35253cb3cbc 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -1163,7 +1163,7 @@ ctf_func_type_info (ctf_file_t *fp, ctf_id_t type, ctf_funcinfo_t *fip)
   if ((dtd = ctf_dynamic_type (fp, type)) == NULL)
     args = (uint32_t *) ((uintptr_t) tp + increment);
   else
-    args = (uint32_t *) dtd->dtd_u.dtu_argv;
+    args = dtd->dtd_u.dtu_argv;
 
   if (fip->ctc_argc != 0 && args[fip->ctc_argc - 1] == 0)
     {
@@ -1174,7 +1174,7 @@ ctf_func_type_info (ctf_file_t *fp, ctf_id_t type, ctf_funcinfo_t *fip)
   return 0;
 }
 
-/* Given a type ID relating to a function type,, return the arguments for the
+/* Given a type ID relating to a function type, return the arguments for the
    function.  */
 
 int
@@ -1200,7 +1200,7 @@ ctf_func_type_args (ctf_file_t *fp, ctf_id_t type, uint32_t argc, ctf_id_t *argv
   if ((dtd = ctf_dynamic_type (fp, type)) == NULL)
     args = (uint32_t *) ((uintptr_t) tp + increment);
   else
-    args = (uint32_t *) dtd->dtd_u.dtu_argv;
+    args = dtd->dtd_u.dtu_argv;
 
   for (argc = MIN (argc, f.ctc_argc); argc != 0; argc--)
     *argv++ = *args++;
-- 
2.27.0.247.g3dff7de930

[PATCH 05/59] libctf, create: add explicit casts for variables' and slices' types

[Nick Alcock]

This is technically unnecessary -- the compiler is quite capable of
doing the range reduction for us -- but it does mean that all
assignments of a ctf_id_t to its final uint32_t representation now have
appropriate explicit casts.

libctf/
	* ctf-create.c (ctf_serialize): Add cast.
	(ctf_add_slice): Likewise.
---
 libctf/ctf-create.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index f91da6ea159..bc0ad802dd2 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -373,7 +373,7 @@ ctf_serialize (ctf_file_t *fp)
       ctf_varent_t *var = &dvarents[i];
 
       ctf_str_add_ref (fp, dvd->dvd_name, &var->ctv_name);
-      var->ctv_type = dvd->dvd_type;
+      var->ctv_type = (uint32_t) dvd->dvd_type;
     }
   assert (i == nvars);
 
@@ -972,7 +972,7 @@ ctf_add_slice (ctf_file_t *fp, uint32_t flag, ctf_id_t ref,
   dtd->dtd_data.ctt_info = CTF_TYPE_INFO (CTF_K_SLICE, flag, 0);
   dtd->dtd_data.ctt_size = clp2 (P2ROUNDUP (ep->cte_bits, CHAR_BIT)
 				 / CHAR_BIT);
-  dtd->dtd_u.dtu_slice.cts_type = ref;
+  dtd->dtd_u.dtu_slice.cts_type = (uint32_t) ref;
   dtd->dtd_u.dtu_slice.cts_bits = ep->cte_bits;
   dtd->dtd_u.dtu_slice.cts_offset = ep->cte_offset;
 
-- 
2.27.0.247.g3dff7de930

[PATCH 06/59] libctf, types: allow ctf_type_reference of dynamic slices

[Nick Alcock]

One spot was missed when we rejigged ctf_update into ctf_serialize and
allowed all operations on dynamic containers: ctf_type_reference of
slices.  A dynamic slice's vlen state is stored in the dtu_slice member,
so fetch it from there.

libctf/
	* ctf-types.c (ctf_type_reference): Add support for dynamic slices.
---
 libctf/ctf-types.c | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index 35253cb3cbc..9c10905ea11 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -680,10 +680,19 @@ ctf_type_reference (ctf_file_t *fp, ctf_id_t type)
       /* Slices store their type in an unusual place.  */
     case CTF_K_SLICE:
       {
+	ctf_dtdef_t *dtd;
 	const ctf_slice_t *sp;
-	ssize_t increment;
-	(void) ctf_get_ctt_size (fp, tp, NULL, &increment);
-	sp = (const ctf_slice_t *) ((uintptr_t) tp + increment);
+
+	if ((dtd = ctf_dynamic_type (ofp, type)) == NULL)
+	  {
+	    ssize_t increment;
+
+	    (void) ctf_get_ctt_size (fp, tp, NULL, &increment);
+	    sp = (const ctf_slice_t *) ((uintptr_t) tp + increment);
+	  }
+	else
+	  sp = &dtd->dtd_u.dtu_slice;
+
 	return sp->cts_type;
       }
     default:
-- 
2.27.0.247.g3dff7de930

[PATCH 07/59] libctf, open: drop unnecessary historical wart around forwards

[Nick Alcock]

When opening, we consider a forward with a kind above the maximum
allowable set of kinds and a forward of kind CTF_K_UNKNOWN to be a
forward to a struct.  Whatever CTF version it was that produced
forwards with no associated kind, it predates anything we can read:
remove this wart.

libctf/
	* ctf-open.c (init_types): Remove typeless CTF_K_FORWARD
	special-casing.
---
 libctf/ctf-open.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index 4daa1e45351..05672f36498 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -692,17 +692,11 @@ init_types (ctf_file_t *fp, ctf_header_t *cth)
       if (vbytes < 0)
 	return ECTF_CORRUPT;
 
+      /* For forward declarations, ctt_type is the CTF_K_* kind for the tag,
+	 so bump that population count too.  */
       if (kind == CTF_K_FORWARD)
-	{
-	  /* For forward declarations, ctt_type is the CTF_K_* kind for the tag,
-	     so bump that population count too.  If ctt_type is unknown, treat
-	     the tag as a struct.  */
+	pop[tp->ctt_type]++;
 
-	  if (tp->ctt_type == CTF_K_UNKNOWN || tp->ctt_type >= CTF_K_MAX)
-	    pop[CTF_K_STRUCT]++;
-	  else
-	    pop[tp->ctt_type]++;
-	}
       tp = (ctf_type_t *) ((uintptr_t) tp + increment + vbytes);
       pop[kind]++;
     }
-- 
2.27.0.247.g3dff7de930

[PATCH 08/59] libctf, create: member names of "" and NULL should be the same

[Nick Alcock]

This matters for the case of unnamed bitfields, whose names are the null
string.  These are special in that they are the only members whose
"names" are allowed to be duplicated in a single struct, but we were
only handling this for the case where name == NULL.  Translate "" to
NULL to help callers.

libctf/
	* ctf-create.c (ctf_add_member_offset): Support names of ""
	as if they were the null pointer.
---
 libctf/ctf-create.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index bc0ad802dd2..848e725ef24 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -1371,6 +1371,9 @@ ctf_add_member_offset (ctf_file_t *fp, ctf_id_t souid, const char *name,
   if (dtd == NULL)
     return (ctf_set_errno (fp, ECTF_BADID));
 
+  if (name != NULL && name[0] == '\0')
+    name = NULL;
+
   kind = LCTF_INFO_KIND (fp, dtd->dtd_data.ctt_info);
   root = LCTF_INFO_ISROOT (fp, dtd->dtd_data.ctt_info);
   vlen = LCTF_INFO_VLEN (fp, dtd->dtd_data.ctt_info);
-- 
2.27.0.247.g3dff7de930

[PATCH 09/59] libctf, create: fix addition of anonymous struct/union members

[Nick Alcock]

A Solaris-era bug causes us to check the offsets of types with no names
against the first such type when ctf_add_type()ing members to a struct
or union.  Members with no names (i.e. anonymous struct/union members)
can appear as many times as you like in a struct/union, so this check
should be skipped in this case.

libctf/
	* ctf-create.c (membcmp)  Skip nameless members.
---
 libctf/ctf-create.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 848e725ef24..5cbcfe0a702 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -1602,6 +1602,11 @@ membcmp (const char *name, ctf_id_t type _libctf_unused_, unsigned long offset,
   ctf_bundle_t *ctb = arg;
   ctf_membinfo_t ctm;
 
+  /* Don't check nameless members (e.g. anonymous structs/unions) against each
+     other.  */
+  if (name[0] == 0)
+    return 0;
+
   if (ctf_member_info (ctb->ctb_file, ctb->ctb_type, name, &ctm) < 0)
     {
       ctf_dprintf ("Conflict due to member %s iteration error: %s.\n", name,
-- 
2.27.0.247.g3dff7de930

[PATCH 10/59] libctf, create: empty dicts are dirty to start with

[Nick Alcock]

Without this, an empty dict that is written out immediately never gets
any content at all: even the header is left empty.

libctf/
	* ctf-create.c (ctf_create): Mark dirty.
---
 libctf/ctf-create.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 5cbcfe0a702..1416d187fd9 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -126,6 +126,7 @@ ctf_create (int *errp)
   fp->ctf_dtoldid = 0;
   fp->ctf_snapshots = 1;
   fp->ctf_snapshot_lu = 0;
+  fp->ctf_flags |= LCTF_DIRTY;
 
   ctf_set_ctl_hashes (fp);
   ctf_setmodel (fp, CTF_MODEL_NATIVE);
-- 
2.27.0.247.g3dff7de930

[PATCH 11/59] libctf, types: support slices of anything terminating in an int

[Nick Alcock]

It is perfectly valid C to say e.g.

typedef u64 int;
struct foo_t
  {
    const volatile u64 wibble:2;
  };

i.e. bitfields have to be integral types, but they can be cv-qualified
integral types or typedefs of same, etc.

This is easy to fix: do a ctf_type_resolve_unsliced() at creation time
to ensure the ultimate type is integral, and ctf_type_resolve() at
lookup time so that if you somehow have e.g. a slice of a typedef of a
slice of a cv-qualified int, we pull the encoding that the topmost slice
is based on out of the subsidiary slice (and then modify it), not out of
the underlying int.  (This last bit is rather academic right now, since
all slices override exactly the same properties of the underlying type,
but it's still the right thing to do.)

libctf/
	* ctf-create.c (ctf_add_slice): Support slices of any kind that
	resolves to an integral type.
	* ctf-types.c (ctf_type_encoding): Resolve the type before
	fishing its encoding out.
---
 libctf/ctf-create.c | 9 ++++++++-
 libctf/ctf-types.c  | 9 +++++++--
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 1416d187fd9..c13b83d4d3b 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -944,6 +944,7 @@ ctf_add_slice (ctf_file_t *fp, uint32_t flag, ctf_id_t ref,
 	       const ctf_encoding_t *ep)
 {
   ctf_dtdef_t *dtd;
+  ctf_id_t resolved_ref = ref;
   ctf_id_t type;
   int kind;
   const ctf_type_t *tp;
@@ -961,7 +962,13 @@ ctf_add_slice (ctf_file_t *fp, uint32_t flag, ctf_id_t ref,
   if (ref != 0 && ((tp = ctf_lookup_by_id (&tmp, ref)) == NULL))
     return CTF_ERR;		/* errno is set for us.  */
 
-  kind = ctf_type_kind_unsliced (tmp, ref);
+  /* Make sure we ultimately point to an integral type.  We also allow slices to
+     point to the unimplemented type, for now, because the compiler can emit
+     such slices, though they're not very much use.  */
+
+  resolved_ref = ctf_type_resolve_unsliced (tmp, ref);
+  kind = ctf_type_kind_unsliced (tmp, resolved_ref);
+
   if ((kind != CTF_K_INTEGER) && (kind != CTF_K_FLOAT) &&
       (kind != CTF_K_ENUM)
       && (ref != 0))
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index 9c10905ea11..f5a1fc0ae12 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -758,9 +758,12 @@ ctf_type_encoding (ctf_file_t *fp, ctf_id_t type, ctf_encoding_t *ep)
 	  {
 	    const ctf_slice_t *slice;
 	    ctf_encoding_t underlying_en;
+	    ctf_id_t underlying;
+
 	    slice = &dtd->dtd_u.dtu_slice;
+	    underlying = ctf_type_resolve (fp, slice->cts_type);
+	    data = ctf_type_encoding (fp, underlying, &underlying_en);
 
-	    data = ctf_type_encoding (fp, slice->cts_type, &underlying_en);
 	    ep->cte_format = underlying_en.cte_format;
 	    ep->cte_offset = slice->cts_offset;
 	    ep->cte_bits = slice->cts_bits;
@@ -792,9 +795,11 @@ ctf_type_encoding (ctf_file_t *fp, ctf_id_t type, ctf_encoding_t *ep)
       {
 	const ctf_slice_t *slice;
 	ctf_encoding_t underlying_en;
+	ctf_id_t underlying;
 
 	slice = (ctf_slice_t *) ((uintptr_t) tp + increment);
-	data = ctf_type_encoding (fp, slice->cts_type, &underlying_en);
+	underlying = ctf_type_resolve (fp, slice->cts_type);
+	data = ctf_type_encoding (fp, underlying, &underlying_en);
 
 	ep->cte_format = underlying_en.cte_format;
 	ep->cte_offset = slice->cts_offset;
-- 
2.27.0.247.g3dff7de930

[PATCH 12/59] libctf, types: ints, floats and typedefs with no name are invalid

[Nick Alcock]

Report them as such, rather than letting ctf_decl_sprintf wrongly
conclude that the printing of zero characters means we are out of
memory.

libctf/
	* ctf-types.c (ctf_type_aname): Return ECTF_CORRUPT if
	ints, floats or typedefs have no name.  Fix comment typo.
---
 libctf/ctf-types.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index f5a1fc0ae12..db42b9e8a90 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -316,7 +316,7 @@ ctf_id_t ctf_lookup_by_rawhash (ctf_file_t *fp, ctf_names_t *np, const char *nam
   return id;
 }
 
-/* Lookup the given type ID and return its name as a new dynamcally-allocated
+/* Lookup the given type ID and return its name as a new dynamically-allocated
    string.  */
 
 char *
@@ -379,6 +379,15 @@ ctf_type_aname (ctf_file_t *fp, ctf_id_t type)
 	    case CTF_K_INTEGER:
 	    case CTF_K_FLOAT:
 	    case CTF_K_TYPEDEF:
+	      /* Integers, floats, and typedefs must always be named types.  */
+
+	      if (name[0] == '\0')
+		{
+		  ctf_set_errno (fp, ECTF_CORRUPT);
+		  ctf_decl_fini (&cd);
+		  return NULL;
+		}
+
 	      ctf_decl_sprintf (&cd, "%s", name);
 	      break;
 	    case CTF_K_POINTER:
-- 
2.27.0.247.g3dff7de930

[PATCH 13/59] libctf, archive: stop ctf_arc_bufopen triggering crazy unmaps

[Nick Alcock]

The archive machinery mmap()s its archives when possible: so it arranges
to do appropriately-sized unmaps by recording the unmap length in the
ctfa_magic value and unmapping that.

This brilliant (horrible) trick works less well when ctf_arc_bufopen is
called with an existing buffer (which might be a readonly mapping).
ctf_arc_bufopen always returns a ctf_archive_t wrapper, so record in
there the necessity to not unmap anything when a bufopen'ed archive is
closed again.

libctf/
	* ctf-impl.h (struct ctf_archive_internal)
	<ctfi_unmap_on_close>: New.
	(ctf_new_archive_internal): Adjust.
	* ctf-archive.c (ctf_new_archive_internal): Likewise.
	Initialize ctfi_unmap_on_close.  Adjust error path.
	(ctf_arc_bufopen): Adjust ctf_new_archive_internal call
	(unmap_on_close is 0).
	(ctf_arc_close): Only unmap if ctfi_unmap_on_close.
	* ctf-open-bfd.c (ctf_fdopen): Adjust.
---
 libctf/ctf-archive.c  | 26 ++++++++++++++++++++------
 libctf/ctf-impl.h     | 10 ++++++----
 libctf/ctf-open-bfd.c |  4 ++--
 3 files changed, 28 insertions(+), 12 deletions(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index ac13d6dd5e9..303c94d97a4 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -343,10 +343,11 @@ search_modent_by_name (const void *key, const void *ent, void *arg)
 
 /* Make a new struct ctf_archive_internal wrapper for a ctf_archive or a
    ctf_file.  Closes ARC and/or FP on error.  Arrange to free the SYMSECT or
-   STRSECT, as needed, on close.  */
+   STRSECT, as needed, on close.  Possibly do not unmap on close.  */
 
 struct ctf_archive_internal *
-ctf_new_archive_internal (int is_archive, struct ctf_archive *arc,
+ctf_new_archive_internal (int is_archive, int unmap_on_close,
+			  struct ctf_archive *arc,
 			  ctf_file_t *fp, const ctf_sect_t *symsect,
 			  const ctf_sect_t *strsect,
 			  int *errp)
@@ -356,7 +357,10 @@ ctf_new_archive_internal (int is_archive, struct ctf_archive *arc,
   if ((arci = calloc (1, sizeof (struct ctf_archive_internal))) == NULL)
     {
       if (is_archive)
-	ctf_arc_close_internal (arc);
+	{
+	  if (unmap_on_close)
+	    ctf_arc_close_internal (arc);
+	}
       else
 	ctf_file_close (fp);
       return (ctf_set_open_errno (errp, errno));
@@ -371,6 +375,7 @@ ctf_new_archive_internal (int is_archive, struct ctf_archive *arc,
   if (strsect)
      memcpy (&arci->ctfi_strsect, strsect, sizeof (struct ctf_sect));
   arci->ctfi_free_symsect = 0;
+  arci->ctfi_unmap_on_close = unmap_on_close;
 
   return arci;
 }
@@ -389,7 +394,13 @@ ctf_arc_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
   if (ctfsect->cts_size > sizeof (uint64_t) &&
       ((*(uint64_t *) ctfsect->cts_data) == CTFA_MAGIC))
     {
-      /* The archive is mmappable, so this operation is trivial.  */
+      /* The archive is mmappable, so this operation is trivial.
+
+	 This buffer is nonmodifiable, so the trick involving mmapping only part
+	 of it and storing the length in the magic number is not applicable: so
+	 record this fact in the archive-wrapper header.  (We cannot record it
+	 in the archive, because the archive may very well be a read-only
+	 mapping.)  */
 
       is_archive = 1;
       arc = (struct ctf_archive *) ctfsect->cts_data;
@@ -404,7 +415,7 @@ ctf_arc_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
 	  return NULL;
 	}
     }
-  return ctf_new_archive_internal (is_archive, arc, fp, symsect, strsect,
+  return ctf_new_archive_internal (is_archive, 0, arc, fp, symsect, strsect,
 				   errp);
 }
 
@@ -481,7 +492,10 @@ ctf_arc_close (ctf_archive_t *arc)
     return;
 
   if (arc->ctfi_is_archive)
-    ctf_arc_close_internal (arc->ctfi_archive);
+    {
+      if (arc->ctfi_unmap_on_close)
+	ctf_arc_close_internal (arc->ctfi_archive);
+    }
   else
     ctf_file_close (arc->ctfi_file);
   if (arc->ctfi_free_symsect)
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index fdd48f0d331..4661aa8c7a9 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -312,6 +312,7 @@ struct ctf_file
 struct ctf_archive_internal
 {
   int ctfi_is_archive;
+  int ctfi_unmap_on_close;
   ctf_file_t *ctfi_file;
   struct ctf_archive *ctfi_archive;
   ctf_sect_t ctfi_symsect;
@@ -443,10 +444,11 @@ extern void ctf_str_rollback (ctf_file_t *, ctf_snapshot_id_t);
 extern void ctf_str_purge_refs (ctf_file_t *);
 extern ctf_strs_writable_t ctf_str_write_strtab (ctf_file_t *);
 
-extern struct ctf_archive_internal *ctf_new_archive_internal
-	(int is_archive, struct ctf_archive *arc,
-	 ctf_file_t *fp, const ctf_sect_t *symsect,
-	 const ctf_sect_t *strsect, int *errp);
+extern struct ctf_archive_internal *
+ctf_new_archive_internal (int is_archive, int unmap_on_close,
+			  struct ctf_archive *, ctf_file_t *,
+			  const ctf_sect_t *symsect,
+			  const ctf_sect_t *strsect, int *errp);
 extern struct ctf_archive *ctf_arc_open_internal (const char *, int *);
 extern void ctf_arc_close_internal (struct ctf_archive *);
 extern void *ctf_set_open_errno (int *, int);
diff --git a/libctf/ctf-open-bfd.c b/libctf/ctf-open-bfd.c
index dafa265f9d8..2d2d572c88f 100644
--- a/libctf/ctf-open-bfd.c
+++ b/libctf/ctf-open-bfd.c
@@ -230,7 +230,7 @@ ctf_fdopen (int fd, const char *filename, const char *target, int *errp)
       fp->ctf_data_mmapped = data;
       fp->ctf_data_mmapped_len = (size_t) st.st_size;
 
-      return ctf_new_archive_internal (0, NULL, fp, NULL, NULL, errp);
+      return ctf_new_archive_internal (0, 1, NULL, fp, NULL, NULL, errp);
     }
 
   if ((nbytes = ctf_pread (fd, &arc_magic, sizeof (arc_magic), 0)) <= 0)
@@ -243,7 +243,7 @@ ctf_fdopen (int fd, const char *filename, const char *target, int *errp)
       if ((arc = ctf_arc_open_internal (filename, errp)) == NULL)
 	return NULL;			/* errno is set for us.  */
 
-      return ctf_new_archive_internal (1, arc, NULL, NULL, NULL, errp);
+      return ctf_new_archive_internal (1, 1, arc, NULL, NULL, NULL, errp);
     }
 
   /* Attempt to open the file with BFD.  We must dup the fd first, since bfd
-- 
2.27.0.247.g3dff7de930

[PATCH 14/59] libctf: having debugging enabled is unlikely

[Nick Alcock]

The deduplicator can emit enormous amounts of debugging output,
so much so that a later commit will introduce a new configure flag
that configures most of it out (and configures it out by default).

It became clear that when this configure flag is on, but debugging is
not enabled via the LIBCTF_DEBUG environment variable, up to 10% of
runtime can be spent on branch mispredictions checking the _libctf_debug
variable.  Mark it unlikely to be set (when it is set, performance is
likely to be the least of your concerns).

libctf/
	* ctf-subr.c (ctf_dprintf): _libctf_debug is unlikely to be set.
---
 libctf/ctf-subr.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-subr.c b/libctf/ctf-subr.c
index 114df843212..a5cde9d6f20 100644
--- a/libctf/ctf-subr.c
+++ b/libctf/ctf-subr.c
@@ -183,7 +183,7 @@ int ctf_getdebug (void)
 _libctf_printflike_ (1, 2)
 void ctf_dprintf (const char *format, ...)
 {
-  if (_libctf_debug)
+  if (_libctf_unlikely_ (_libctf_debug))
     {
       va_list alist;
 
-- 
2.27.0.247.g3dff7de930

[PATCH 15/59] libctf: add ctf_type_name_raw

[Nick Alcock]

We already have a function ctf_type_aname_raw, which returns the raw
name of a type with no decoration for structures or arrays or anything
like that: just the underlying name of whatever it is that's being
ultimately pointed at.

But this can be inconvenient to use, becauswe it always allocates new
storage for the string and copies it in, so it can potentially fail.
Add ctf_type_name_raw, which just returns the string directly out of
libctf's guts: it will live until the ctf_file_t is closed (if we later
gain the ability to remove types from writable dicts, it will live as
long as the type lives).

Reimplement ctf_type_aname_raw in terms of it.

include/
	* ctf-api.c (ctf_type_name_raw): New.

libctf/
	* ctf-types.c (ctf_type_name_raw): New.
	(ctf_type_aname_raw): Reimplement accordingly.
---
 include/ctf-api.h  |  1 +
 libctf/ctf-types.c | 23 +++++++++++++++++------
 libctf/libctf.ver  |  1 +
 3 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 2e3e28b840c..363b5c258ca 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -324,6 +324,7 @@ extern char *ctf_type_aname (ctf_file_t *, ctf_id_t);
 extern char *ctf_type_aname_raw (ctf_file_t *, ctf_id_t);
 extern ssize_t ctf_type_lname (ctf_file_t *, ctf_id_t, char *, size_t);
 extern char *ctf_type_name (ctf_file_t *, ctf_id_t, char *, size_t);
+extern const char *ctf_type_name_raw (ctf_file_t *, ctf_id_t);
 extern ssize_t ctf_type_size (ctf_file_t *, ctf_id_t);
 extern ssize_t ctf_type_align (ctf_file_t *, ctf_id_t);
 extern int ctf_type_kind (ctf_file_t *, ctf_id_t);
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index db42b9e8a90..ce3890c33a6 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -472,19 +472,30 @@ ctf_type_name (ctf_file_t *fp, ctf_id_t type, char *buf, size_t len)
   return (rv >= 0 && (size_t) rv < len ? buf : NULL);
 }
 
-/* Lookup the given type ID and return its raw, unadorned, undecorated name as a
-   new dynamcally-allocated string.  */
+/* Lookup the given type ID and return its raw, unadorned, undecorated name.
+   The name will live as long as its ctf_file_t does.  */
 
-char *
-ctf_type_aname_raw (ctf_file_t *fp, ctf_id_t type)
+const char *
+ctf_type_name_raw (ctf_file_t *fp, ctf_id_t type)
 {
   const ctf_type_t *tp;
 
   if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
     return NULL;		/* errno is set for us.  */
 
-  if (ctf_strraw (fp, tp->ctt_name) != NULL)
-    return strdup (ctf_strraw (fp, tp->ctt_name));
+  return ctf_strraw (fp, tp->ctt_name);
+}
+
+/* Lookup the given type ID and return its raw, unadorned, undecorated name as a
+   new dynamically-allocated string.  */
+
+char *
+ctf_type_aname_raw (ctf_file_t *fp, ctf_id_t type)
+{
+  const char *name = ctf_type_name_raw (fp, type);
+
+  if (name != NULL)
+    return strdup (name);
 
   return NULL;
 }
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index aad304bc0d9..30a0b087bd6 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -57,6 +57,7 @@ LIBCTF_1.0 {
 	ctf_type_resolve;
 	ctf_type_lname;
 	ctf_type_name;
+	ctf_type_name_raw;
 	ctf_type_aname;
 	ctf_type_aname_raw;
 	ctf_type_size;
-- 
2.27.0.247.g3dff7de930

[PATCH 16/59] libctf: add ctf_type_kind_forwarded

[Nick Alcock]

This is just like ctf_type_kind, except that forwards get the
type of the thing being pointed to rather than CTF_K_FORWARD.

include/
	* ctf-api.h (ctf_type_kind_forwarded): New.
libctf/
	* ctf-types.c (ctf_type_kind_forwarded): New.
---
 include/ctf-api.h  |  1 +
 libctf/ctf-types.c | 20 ++++++++++++++++++++
 libctf/libctf.ver  |  1 +
 3 files changed, 22 insertions(+)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 363b5c258ca..87446a5f7d1 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -328,6 +328,7 @@ extern const char *ctf_type_name_raw (ctf_file_t *, ctf_id_t);
 extern ssize_t ctf_type_size (ctf_file_t *, ctf_id_t);
 extern ssize_t ctf_type_align (ctf_file_t *, ctf_id_t);
 extern int ctf_type_kind (ctf_file_t *, ctf_id_t);
+extern int ctf_type_kind_forwarded (ctf_file_t *, ctf_id_t);
 extern ctf_id_t ctf_type_reference (ctf_file_t *, ctf_id_t);
 extern ctf_id_t ctf_type_pointer (ctf_file_t *, ctf_id_t);
 extern int ctf_type_encoding (ctf_file_t *, ctf_id_t, ctf_encoding_t *);
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index ce3890c33a6..d8f848cf893 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -677,6 +677,26 @@ ctf_type_kind (ctf_file_t *fp, ctf_id_t type)
   return kind;
 }
 
+/* Return the kind of this type, except, for forwards, return the kind of thing
+   this is a forward to.  */
+int
+ctf_type_kind_forwarded (ctf_file_t *fp, ctf_id_t type)
+{
+  int kind;
+  const ctf_type_t *tp;
+
+  if ((kind = ctf_type_kind (fp, type)) < 0)
+    return -1;			/* errno is set for us.  */
+
+  if (kind != CTF_K_FORWARD)
+    return kind;
+
+  if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+    return -1;			/* errno is set for us.  */
+
+  return tp->ctt_type;
+}
+
 /* If the type is one that directly references another type (such as POINTER),
    then return the ID of the type to which it refers.  */
 
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 30a0b087bd6..b8c5133d3f9 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -63,6 +63,7 @@ LIBCTF_1.0 {
 	ctf_type_size;
 	ctf_type_align;
 	ctf_type_kind;
+	ctf_type_kind_forwarded;
 	ctf_type_reference;
 	ctf_type_pointer;
 	ctf_type_encoding;
-- 
2.27.0.247.g3dff7de930

[PATCH 17/59] libctf: add ctf_member_count

[Nick Alcock]

This returns the number of members in a struct or union, or the number
of enumerations in an enum.  (This was only available before now by
iterating across every member, but it can be returned much faster than
that.)

include/
	* ctf-api.h (ctf_member_count): New.

libctf/
	* ctf-types.c (ctf_member_count): New.
	* libctf.ver: New public function.
---
 include/ctf-api.h  |  1 +
 libctf/ctf-types.c | 24 ++++++++++++++++++++++++
 libctf/libctf.ver  |  1 +
 3 files changed, 26 insertions(+)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 87446a5f7d1..7cdf4a56235 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -349,6 +349,7 @@ extern const char *ctf_label_get (ctf_file_t *);
 extern const char *ctf_label_topmost (ctf_file_t *);
 extern int ctf_label_info (ctf_file_t *, const char *, ctf_lblinfo_t *);
 
+extern int ctf_member_count (ctf_file_t *, ctf_id_t);
 extern int ctf_member_iter (ctf_file_t *, ctf_id_t, ctf_member_f *, void *);
 extern int ctf_enum_iter (ctf_file_t *, ctf_id_t, ctf_enum_f *, void *);
 extern int ctf_type_iter (ctf_file_t *, ctf_type_f *, void *);
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index d8f848cf893..93967c6aa8d 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -963,6 +963,30 @@ ctf_type_compat (ctf_file_t *lfp, ctf_id_t ltype,
     }
 }
 
+/* Return the number of members in a STRUCT or UNION, or the number of
+   enumerators in an ENUM.  */
+
+int
+ctf_member_count (ctf_file_t *fp, ctf_id_t type)
+{
+  ctf_file_t *ofp = fp;
+  const ctf_type_t *tp;
+  uint32_t kind;
+
+  if ((type = ctf_type_resolve (fp, type)) == CTF_ERR)
+    return -1;			/* errno is set for us.  */
+
+  if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+    return -1;			/* errno is set for us.  */
+
+  kind = LCTF_INFO_KIND (fp, tp->ctt_info);
+
+  if (kind != CTF_K_STRUCT && kind != CTF_K_UNION && kind != CTF_K_ENUM)
+    return (ctf_set_errno (ofp, ECTF_NOTSUE));
+
+  return LCTF_INFO_VLEN (fp, tp->ctt_info);
+}
+
 /* Return the type and offset for a given member of a STRUCT or UNION.  */
 
 int
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index b8c5133d3f9..77b380ca991 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -73,6 +73,7 @@ LIBCTF_1.0 {
 
 	ctf_member_info;
 	ctf_array_info;
+	ctf_member_count;
 
 	ctf_enum_name;
 	ctf_enum_value;
-- 
2.27.0.247.g3dff7de930

[PATCH 18/59] libctf: add ctf_archive_count

[Nick Alcock]

Another count that was otherwise unavailable without doing expensive
operations.

include/
	* ctf-api.h (ctf_archive_count): New.

libctf/
	* ctf-archive.c (ctf_archive_count): New.
	* libctf.ver: New public function.
---
 include/ctf-api.h    |  1 +
 libctf/ctf-archive.c | 10 ++++++++++
 libctf/libctf.ver    |  1 +
 3 files changed, 12 insertions(+)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 7cdf4a56235..fb797a3346d 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -276,6 +276,7 @@ extern ctf_file_t *ctf_arc_open_by_name_sections (const ctf_archive_t *,
 						  const ctf_sect_t *,
 						  const ctf_sect_t *,
 						  const char *, int *);
+extern size_t ctf_archive_count (const ctf_archive_t *);
 
 /* The next functions return or close real CTF files, or write out CTF archives,
    not opaque containers around either.  */
diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 303c94d97a4..1cfe40f6e3a 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -624,6 +624,16 @@ ctf_arc_open_by_offset (const struct ctf_archive *arc,
   return fp;
 }
 
+/* Return the number of members in an archive.  */
+size_t
+ctf_archive_count (const ctf_archive_t *wrapper)
+{
+  if (!wrapper->ctfi_is_archive)
+    return 1;
+
+  return wrapper->ctfi_archive->ctfa_nfiles;
+}
+
 /* Raw iteration over all CTF files in an archive.  We pass the raw data for all
    CTF files in turn to the specified callback function.  */
 static int
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 77b380ca991..375dee8fc77 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -135,6 +135,7 @@ LIBCTF_1.0 {
 	ctf_arc_close;
 	ctf_arc_open_by_name;
 	ctf_arc_open_by_name_sections;
+	ctf_archive_count;
 	ctf_archive_iter;
 	ctf_archive_raw_iter;
 	ctf_get_arc;
-- 
2.27.0.247.g3dff7de930

[PATCH 19/59] libctf: fix __extension__ with non-GNU C compilers

[Nick Alcock]

We forgot to #define __extension__ to nothing in this case.

libctf/
	* ctf-impl.h [!__GNUC__] (__extension__): Define to nothing.
---
 libctf/ctf-impl.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 4661aa8c7a9..f832c71b97c 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -65,6 +65,7 @@ extern "C"
 #define _libctf_unlikely_(x) (x)
 #define _libctf_unused_
 #define _libctf_malloc_
+#define __extension__
 
 #endif
 
-- 
2.27.0.247.g3dff7de930

[PATCH 20/59] libctf: add new dynhash functions

[Nick Alcock]

Future commits will use these.

ctf_dynhash_elements: count elements in a dynhash
ctf_dynhash_lookup_kv: look up and return pointers to the original key
                       and value in a dynhash (the only way of getting
                       a reference to the original key)
ctf_dynhash_iter_find: iterate until an item is found, then return its
                       key
ctf_dynhash_cinsert: insert a const key / value into a dynhash (a thim
                     wrapper in a new header dedicated to inline
                     functions).

As with the rest of ctf_dynhash, this is not public API.  No impact
on existing callers is expected.

libctf/
	* ctf-inlines.h: New file.
	* ctf-impl.h: Include it.
	(ctf_hash_iter_find_f): New typedef.
	(ctf_dynhash_elements): New.
	(ctf_dynhash_lookup_kv): New.
	(ctf_dynhash_iter_find): New.
	* ctf-hash.c (ctf_dynhash_lookup_kv): New.
	(ctf_traverse_find_cb_arg_t): New.
	(ctf_hashtab_traverse_find): New.
	(ctf_dynhash_iter_find): New.
	(ctf_dynhash_elements): New.
---
 libctf/ctf-hash.c    | 55 ++++++++++++++++++++++++++++++++++++++++++++
 libctf/ctf-impl.h    |  8 +++++++
 libctf/ctf-inlines.h | 45 ++++++++++++++++++++++++++++++++++++
 3 files changed, 108 insertions(+)
 create mode 100644 libctf/ctf-inlines.h

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 71c1f8e4e21..4696fcb2d43 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -218,6 +218,12 @@ ctf_dynhash_empty (ctf_dynhash_t *hp)
   htab_empty (hp->htab);
 }
 
+size_t
+ctf_dynhash_elements (ctf_dynhash_t *hp)
+{
+  return htab_elements (hp->htab);
+}
+
 void *
 ctf_dynhash_lookup (ctf_dynhash_t *hp, const void *key)
 {
@@ -231,6 +237,26 @@ ctf_dynhash_lookup (ctf_dynhash_t *hp, const void *key)
   return NULL;
 }
 
+/* TRUE/FALSE return.  */
+int
+ctf_dynhash_lookup_kv (ctf_dynhash_t *hp, const void *key,
+		       const void **orig_key, void **value)
+{
+  ctf_helem_t **slot;
+
+  slot = ctf_hashtab_lookup (hp->htab, key, NO_INSERT);
+
+  if (slot)
+    {
+      if (orig_key)
+	*orig_key = (*slot)->key;
+      if (value)
+	*value = (*slot)->value;
+      return 1;
+    }
+  return 0;
+}
+
 typedef struct ctf_traverse_cb_arg
 {
   ctf_hash_iter_f fun;
@@ -254,6 +280,35 @@ ctf_dynhash_iter (ctf_dynhash_t *hp, ctf_hash_iter_f fun, void *arg_)
   htab_traverse (hp->htab, ctf_hashtab_traverse, &arg);
 }
 
+typedef struct ctf_traverse_find_cb_arg
+{
+  ctf_hash_iter_find_f fun;
+  void *arg;
+  void *found_key;
+} ctf_traverse_find_cb_arg_t;
+
+static int
+ctf_hashtab_traverse_find (void **slot, void *arg_)
+{
+  ctf_helem_t *helem = *((ctf_helem_t **) slot);
+  ctf_traverse_find_cb_arg_t *arg = (ctf_traverse_find_cb_arg_t *) arg_;
+
+  if (arg->fun (helem->key, helem->value, arg->arg))
+    {
+      arg->found_key = helem->key;
+      return 0;
+    }
+  return 1;
+}
+
+void *
+ctf_dynhash_iter_find (ctf_dynhash_t *hp, ctf_hash_iter_find_f fun, void *arg_)
+{
+  ctf_traverse_find_cb_arg_t arg = { fun, arg_, NULL };
+  htab_traverse (hp->htab, ctf_hashtab_traverse_find, &arg);
+  return arg.found_key;
+}
+
 typedef struct ctf_traverse_remove_cb_arg
 {
   struct htab *htab;
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index f832c71b97c..3e097795bc6 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -382,6 +382,7 @@ typedef void (*ctf_hash_free_fun) (void *);
 
 typedef void (*ctf_hash_iter_f) (void *key, void *value, void *arg);
 typedef int (*ctf_hash_iter_remove_f) (void *key, void *value, void *arg);
+typedef int (*ctf_hash_iter_find_f) (void *key, void *value, void *arg);
 
 extern ctf_hash_t *ctf_hash_create (unsigned long, ctf_hash_fun, ctf_hash_eq_fun);
 extern int ctf_hash_insert_type (ctf_hash_t *, ctf_file_t *, uint32_t, uint32_t);
@@ -394,12 +395,17 @@ extern ctf_dynhash_t *ctf_dynhash_create (ctf_hash_fun, ctf_hash_eq_fun,
 					  ctf_hash_free_fun, ctf_hash_free_fun);
 extern int ctf_dynhash_insert (ctf_dynhash_t *, void *, void *);
 extern void ctf_dynhash_remove (ctf_dynhash_t *, const void *);
+extern size_t ctf_dynhash_elements (ctf_dynhash_t *);
 extern void ctf_dynhash_empty (ctf_dynhash_t *);
 extern void *ctf_dynhash_lookup (ctf_dynhash_t *, const void *);
+extern int ctf_dynhash_lookup_kv (ctf_dynhash_t *, const void *key,
+				  const void **orig_key, void **value);
 extern void ctf_dynhash_destroy (ctf_dynhash_t *);
 extern void ctf_dynhash_iter (ctf_dynhash_t *, ctf_hash_iter_f, void *);
 extern void ctf_dynhash_iter_remove (ctf_dynhash_t *, ctf_hash_iter_remove_f,
 				     void *);
+extern void *ctf_dynhash_iter_find (ctf_dynhash_t *, ctf_hash_iter_find_f,
+				    void *);
 
 #define	ctf_list_prev(elem)	((void *)(((ctf_list_t *)(elem))->l_prev))
 #define	ctf_list_next(elem)	((void *)(((ctf_list_t *)(elem))->l_next))
@@ -492,6 +498,8 @@ extern const char _CTF_NULLSTR[];	/* empty string */
 extern int _libctf_version;	/* library client version */
 extern int _libctf_debug;	/* debugging messages enabled */
 
+#include "ctf-inlines.h"
+
 #ifdef	__cplusplus
 }
 #endif
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
new file mode 100644
index 00000000000..ad74b39f539
--- /dev/null
+++ b/libctf/ctf-inlines.h
@@ -0,0 +1,45 @@
+/* Inline functions.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+
+   This file is part of libctf.
+
+   libctf is free software; you can redistribute it and/or modify it under
+   the terms of the GNU General Public License as published by the Free
+   Software Foundation; either version 3, or (at your option) any later
+   version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+   See the GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; see the file COPYING.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef	_CTF_INLINES_H
+#define	_CTF_INLINES_H
+
+#ifdef	__cplusplus
+extern "C"
+{
+#endif
+
+#include "config.h"
+
+#ifndef _libctf_malloc_
+#error "ctf-inlines.h" should not be included directly: include "ctf-impl.h".
+#endif
+
+
+static inline int
+ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
+{
+  return ctf_dynhash_insert (h, (void *) k, (void *) v);
+}
+
+#ifdef	__cplusplus
+}
+#endif
+
+#endif /* _CTF_INLINES_H */
-- 
2.27.0.247.g3dff7de930

[PATCH 21/59] libctf, hash: improve insertion of existing keys into dynhashes

[Nick Alcock]

Right now, if you insert a key/value pair into a dynhash, the old slot's
key is freed and the new one always assigned.  This seemed sane to me
when I wrote it, but I got it wrong time and time again.  It's much
less confusing to free the key passed in: if a key-freeing function
was passed, you are asserting that the dynhash owns the key in any
case, so if you pass in a key it is always buggy to assume it sticks
around.  Freeing the old key means that you can't even safely look up a
key from out of a dynhash and hold on to it, because some other matching
key might force it to be freed at any time.

In the new model, you can always get a key out of a dynhash with
ctf_dynhash_lookup_kv and hang on to it until the kv-pair is actually
deleted from the dynhash.  In the old model the pointer to the key might
be freed at any time if a matching key was inserted.

libctf/
	* ctf-hash.c (ctf_hashtab_insert): Free the key passed in if
	there is a key-freeing function and the key already exists.
---
 libctf/ctf-hash.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 4696fcb2d43..1c37d7515b4 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -171,15 +171,15 @@ ctf_hashtab_insert (struct htab *htab, void *key, void *value,
       *slot = malloc (sizeof (ctf_helem_t));
       if (!*slot)
 	return NULL;
+      (*slot)->key = key;
     }
   else
     {
       if (key_free)
-	  key_free ((*slot)->key);
+	  key_free (key);
       if (value_free)
 	  value_free ((*slot)->value);
     }
-  (*slot)->key = key;
   (*slot)->value = value;
   return *slot;
 }
-- 
2.27.0.247.g3dff7de930

[PATCH 22/59] libctf, hash: save per-item space when no key/item freeing function

[Nick Alcock]

The libctf dynhash hashtab abstraction supports per-hashtab arbitrary
key/item freeing functions -- but it also has a constant slot type that
holds both key and value requested by the user, so it needs to use its
own freeing function to free that -- and it has nowhere to store the
freeing functions the caller requested.

So it copies them into every hash item, bloating every slot, even though
all items in a given hash table must have the same key and value freeing
functions.

So point back to the owner using a back-pointer, but don't even spend
space in the item or the hashtab allocating those freeing functions
unless necessary: if none are needed, we can simply arrange to not pass
in ctf_dynhash_item_free as a del_f to hashtab_create_alloc, and none of
those fields will ever be accessed.

The only downside is that this makes the code sensitive to the order of
fields in the ctf_helem_t and ctf_hashtab_t: but the deduplicator
allocates so many hash tables that doing this alone cuts memory usage
during deduplication by about 10%.  (libiberty hashtab itself has a lot
of per-hashtab bloat: in the future we might trim that down, or make a
trimmer version.)

libctf/
	* ctf-hash.c (ctf_helem_t) <key_free>: Remove.
	<value_free>: Likewise.
	<owner>: New.
	(ctf_dynhash_item_free): Indirect through the owner.
	(ctf_dynhash_create): Only pass in ctf_dynhash_item_free and
	allocate space for the key_free and value_free fields fields
	if necessary.
	(ctf_hashtab_insert): Likewise.  Fix OOM errno value.
	(ctf_dynhash_insert): Only access ctf_hashtab's key_free and
	value_free if they will exist.  Set the slot's owner, but only
	if it exists.
	(ctf_dynhash_remove): Adjust.
---
 libctf/ctf-hash.c | 68 ++++++++++++++++++++++++++++++++---------------
 1 file changed, 47 insertions(+), 21 deletions(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 1c37d7515b4..caefd99c37f 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -31,14 +31,19 @@
    not support removal.  These can be implemented by the same underlying hashmap
    if you wish.  */
 
+/* The helem is used for general key/value mappings in both the ctf_hash and
+   ctf_dynhash: the owner may not have space allocated for it, and will be
+   garbage (not NULL!) in that case.  */
+
 typedef struct ctf_helem
 {
   void *key;			 /* Either a pointer, or a coerced ctf_id_t.  */
   void *value;			 /* The value (possibly a coerced int).  */
-  ctf_hash_free_fun key_free;
-  ctf_hash_free_fun value_free;
+  ctf_dynhash_t *owner;          /* The hash that owns us.  */
 } ctf_helem_t;
 
+/* Equally, the key_free and value_free may not exist.  */
+
 struct ctf_dynhash
 {
   struct htab *htab;
@@ -106,17 +111,17 @@ ctf_hash_eq_type_mapping_key (const void *a, const void *b)
 
 /* The dynhash, used for hashes whose size is not known at creation time. */
 
-/* Free a single ctf_helem.  */
+/* Free a single ctf_helem with arbitrary key/value functions.  */
 
 static void
 ctf_dynhash_item_free (void *item)
 {
   ctf_helem_t *helem = item;
 
-  if (helem->key_free && helem->key)
-    helem->key_free (helem->key);
-  if (helem->value_free && helem->value)
-    helem->value_free (helem->value);
+  if (helem->owner->key_free && helem->key)
+    helem->owner->key_free (helem->key);
+  if (helem->owner->value_free && helem->value)
+    helem->owner->value_free (helem->value);
   free (helem);
 }
 
@@ -125,21 +130,31 @@ ctf_dynhash_create (ctf_hash_fun hash_fun, ctf_hash_eq_fun eq_fun,
                     ctf_hash_free_fun key_free, ctf_hash_free_fun value_free)
 {
   ctf_dynhash_t *dynhash;
+  htab_del del = ctf_dynhash_item_free;
 
-  dynhash = malloc (sizeof (ctf_dynhash_t));
+  if (key_free || value_free)
+    dynhash = malloc (sizeof (ctf_dynhash_t));
+  else
+    dynhash = malloc (offsetof (ctf_dynhash_t, key_free));
   if (!dynhash)
     return NULL;
 
-  /* 7 is arbitrary and untested for now..  */
+  if (key_free == NULL && value_free == NULL)
+    del = free;
+
+  /* 7 is arbitrary and untested for now.  */
   if ((dynhash->htab = htab_create_alloc (7, (htab_hash) hash_fun, eq_fun,
-                                          ctf_dynhash_item_free, xcalloc, free)) == NULL)
+					  del, xcalloc, free)) == NULL)
     {
       free (dynhash);
       return NULL;
     }
 
-  dynhash->key_free = key_free;
-  dynhash->value_free = value_free;
+  if (key_free || value_free)
+    {
+      dynhash->key_free = key_free;
+      dynhash->value_free = value_free;
+    }
 
   return dynhash;
 }
@@ -162,13 +177,18 @@ ctf_hashtab_insert (struct htab *htab, void *key, void *value,
 
   if (!slot)
     {
-      errno = -ENOMEM;
+      errno = ENOMEM;
       return NULL;
     }
 
   if (!*slot)
     {
-      *slot = malloc (sizeof (ctf_helem_t));
+      /* Only spend space on the owner if we're going to use it: if there is a
+	 key or value freeing function.  */
+      if (key_free || value_free)
+	*slot = malloc (sizeof (ctf_helem_t));
+      else
+	*slot = malloc (offsetof (ctf_helem_t, owner));
       if (!*slot)
 	return NULL;
       (*slot)->key = key;
@@ -188,19 +208,25 @@ int
 ctf_dynhash_insert (ctf_dynhash_t *hp, void *key, void *value)
 {
   ctf_helem_t *slot;
+  ctf_hash_free_fun key_free = NULL, value_free = NULL;
 
+  if (hp->htab->del_f == ctf_dynhash_item_free)
+    {
+      key_free = hp->key_free;
+      value_free = hp->value_free;
+    }
   slot = ctf_hashtab_insert (hp->htab, key, value,
-			     hp->key_free, hp->value_free);
+			     key_free, value_free);
 
   if (!slot)
     return errno;
 
-  /* We need to keep the key_free and value_free around in each item because the
-     del function has no visibility into the hash as a whole, only into the
-     individual items.  */
+  /* Keep track of the owner, so that the del function can get at the key_free
+     and value_free functions.  Only do this if one of those functions is set:
+     if not, the owner is not even present in the helem.  */
 
-  slot->key_free = hp->key_free;
-  slot->value_free = hp->value_free;
+  if (key_free || value_free)
+    slot->owner = hp;
 
   return 0;
 }
@@ -208,7 +234,7 @@ ctf_dynhash_insert (ctf_dynhash_t *hp, void *key, void *value)
 void
 ctf_dynhash_remove (ctf_dynhash_t *hp, const void *key)
 {
-  ctf_helem_t hep = { (void *) key, NULL, NULL, NULL };
+  ctf_helem_t hep = { (void *) key, NULL, NULL };
   htab_remove_elt (hp->htab, &hep);
 }
 
-- 
2.27.0.247.g3dff7de930

[PATCH 23/59] libctf, hash: introduce the ctf_dynset

[Nick Alcock]

There are many places in the deduplicator which use hashtables as tiny
sets: keys with no value (and usually, but not always, no freeing
function) often with only one or a few members.  For each of these, even
after the last change to not store the freeing functions, we are storing
a little malloced block for each item just to track the key/value pair,
and a little malloced block for the hash table itself just to track the
freeing function because we can't use libiberty hashtab's freeing
function because we are using that to free the little malloced per-item
block.

If we only have a key, we don't need any of that: we can ditch the
per-malloced block because we don't have a value, and we can ditch the
per-hashtab structure because we don't need to independently track the
freeing functions since libiberty hashtab is doing it for us.  That
means we don't need an owner field in the (now nonexistent) item block
either.

Roughly speaking, this datatype saves about 25% in time and 20% in peak
memory usage for normal links, even fairly big ones.  So this might seem
redundant, but it's really worth it.

Instead of a _lookup function, a dynset has two distinct functions:
ctf_dynset_exists, which returns true or false and an optional pointer
to the set member, and ctf_dynhash_lookup_any, which is used if all
members of the set are expected to be equivalent and we just want *any*
member and we don't care which one.

There is no iterator in this set of functions, not because we don't
iterate over dynset members -- we do, a lot -- but because the iterator
here is a member of an entirely new family of much more convenient
iteration functions, introduced in the next commit.

libctf/
	* ctf-hash.c (ctf_dynset_eq_string): New.
	(ctf_dynset_create): New.
	(DYNSET_EMPTY_ENTRY_REPLACEMENT): New.
	(DYNSET_DELETED_ENTRY_REPLACEMENT): New.
	(key_to_internal): New.
	(internal_to_key): New.
	(ctf_dynset_insert): New.
	(ctf_dynset_remove): New.
	(ctf_dynset_destroy): New.
	(ctf_dynset_lookup): New.
	(ctf_dynset_exists): New.
	(ctf_dynset_lookup_any): New.
	(ctf_hash_insert_type): Coding style.
	(ctf_hash_define_type): Likewise.

	* ctf-impl.h (ctf_dynset_t): New.
	(ctf_dynset_eq_string): New.
	(ctf_dynset_create): New.
	(ctf_dynset_insert): New.
	(ctf_dynset_remove): New.
	(ctf_dynset_destroy): New.
	(ctf_dynset_lookup): New.
	(ctf_dynset_exists): New.
	(ctf_dynset_lookup_any): New.
	* ctf-inlines.h (ctf_dynset_cinsert): New.
---
 libctf/ctf-hash.c    | 168 ++++++++++++++++++++++++++++++++++++++++---
 libctf/ctf-impl.h    |  13 ++++
 libctf/ctf-inlines.h |   6 ++
 3 files changed, 176 insertions(+), 11 deletions(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index caefd99c37f..7eba494a51d 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -22,14 +22,21 @@
 #include "libiberty.h"
 #include "hashtab.h"
 
-/* We have two hashtable implementations: one, ctf_dynhash_*(), is an interface to
-   a dynamically-expanding hash with unknown size that should support addition
-   of large numbers of items, and removal as well, and is used only at
-   type-insertion time; the other, ctf_dynhash_*(), is an interface to a
-   fixed-size hash from const char * -> ctf_id_t with number of elements
-   specified at creation time, that should support addition of items but need
-   not support removal.  These can be implemented by the same underlying hashmap
-   if you wish.  */
+/* We have three hashtable implementations:
+
+   - ctf_hash_* is an interface to a fixed-size hash from const char * ->
+     ctf_id_t with number of elements specified at creation time, that should
+     support addition of items but need not support removal.
+
+   - ctf_dynhash_* is an interface to a dynamically-expanding hash with
+     unknown size that should support addition of large numbers of items, and
+     removal as well, and is used only at type-insertion time and during
+     linking.
+
+   - ctf_dynset_* is an interface to a dynamically-expanding hash that contains
+     only keys: no values.
+
+   These can be implemented by the same underlying hashmap if you wish.  */
 
 /* The helem is used for general key/value mappings in both the ctf_hash and
    ctf_dynhash: the owner may not have space allocated for it, and will be
@@ -51,7 +58,7 @@ struct ctf_dynhash
   ctf_hash_free_fun value_free;
 };
 
-/* Hash functions. */
+/* Hash and eq functions for the dynhash and hash. */
 
 unsigned int
 ctf_hash_integer (const void *ptr)
@@ -109,6 +116,16 @@ ctf_hash_eq_type_mapping_key (const void *a, const void *b)
     && (key_a->cltm_idx == key_b->cltm_idx);
 }
 
+
+/* Hash and eq functions for the dynset.  Most of these can just use the
+   underlying hashtab functions directly.   */
+
+int
+ctf_dynset_eq_string (const void *a, const void *b)
+{
+  return !strcmp((const char *) a, (const char *) b);
+}
+
 /* The dynhash, used for hashes whose size is not known at creation time. */
 
 /* Free a single ctf_helem with arbitrary key/value functions.  */
@@ -369,6 +386,135 @@ ctf_dynhash_destroy (ctf_dynhash_t *hp)
   free (hp);
 }
 
+/* The dynset, used for sets of keys with no value.  The implementation of this
+   can be much simpler, because without a value the slot can simply be the
+   stored key, which means we don't need to store the freeing functions and the
+   dynset itself is just a htab.  */
+
+ctf_dynset_t *
+ctf_dynset_create (htab_hash hash_fun, htab_eq eq_fun,
+		   ctf_hash_free_fun key_free)
+{
+  /* 7 is arbitrary and untested for now.  */
+  return (ctf_dynset_t *) htab_create_alloc (7, (htab_hash) hash_fun, eq_fun,
+					     key_free, xcalloc, free);
+}
+
+/* The dynset has one complexity: the underlying implementation reserves two
+   values for internal hash table implementation details (empty versus deleted
+   entries).  These values are otherwise very useful for pointers cast to ints,
+   so transform the ctf_dynset_inserted value to allow for it.  (This
+   introduces an ambiguity in that one can no longer store these two values in
+   the dynset, but if we pick high enough values this is very unlikely to be a
+   problem.)
+
+   We leak this implementation detail to the freeing functions on the grounds
+   that any use of these functions is overwhelmingly likely to be in sets using
+   real pointers, which will be unaffected.  */
+
+#define DYNSET_EMPTY_ENTRY_REPLACEMENT ((void *) (uintptr_t) -64)
+#define DYNSET_DELETED_ENTRY_REPLACEMENT ((void *) (uintptr_t) -63)
+
+static void *
+key_to_internal (const void *key)
+{
+  if (key == HTAB_EMPTY_ENTRY)
+    return DYNSET_EMPTY_ENTRY_REPLACEMENT;
+  else if (key == HTAB_DELETED_ENTRY)
+    return DYNSET_DELETED_ENTRY_REPLACEMENT;
+
+  return (void *) key;
+}
+
+static void *
+internal_to_key (const void *internal)
+{
+  if (internal == DYNSET_EMPTY_ENTRY_REPLACEMENT)
+    return HTAB_EMPTY_ENTRY;
+  else if (internal == DYNSET_DELETED_ENTRY_REPLACEMENT)
+    return HTAB_DELETED_ENTRY;
+  return (void *) internal;
+}
+
+int
+ctf_dynset_insert (ctf_dynset_t *hp, void *key)
+{
+  struct htab *htab = (struct htab *) hp;
+  void **slot;
+
+  slot = htab_find_slot (htab, key, INSERT);
+
+  if (!slot)
+    {
+      errno = ENOMEM;
+      return -errno;
+    }
+
+  if (*slot)
+    {
+      if (htab->del_f)
+	(*htab->del_f) (*slot);
+    }
+
+  *slot = key_to_internal (key);
+
+  return 0;
+}
+
+void
+ctf_dynset_remove (ctf_dynset_t *hp, const void *key)
+{
+  htab_remove_elt ((struct htab *) hp, key_to_internal (key));
+}
+
+void
+ctf_dynset_destroy (ctf_dynset_t *hp)
+{
+  if (hp != NULL)
+    htab_delete ((struct htab *) hp);
+}
+
+void *
+ctf_dynset_lookup (ctf_dynset_t *hp, const void *key)
+{
+  void **slot = htab_find_slot ((struct htab *) hp,
+				key_to_internal (key), NO_INSERT);
+
+  if (slot)
+    return internal_to_key (*slot);
+  return NULL;
+}
+
+/* TRUE/FALSE return.  */
+int
+ctf_dynset_exists (ctf_dynset_t *hp, const void *key, const void **orig_key)
+{
+  void **slot = htab_find_slot ((struct htab *) hp,
+				key_to_internal (key), NO_INSERT);
+
+  if (orig_key && slot)
+    *orig_key = internal_to_key (*slot);
+  return (slot != NULL);
+}
+
+/* Look up a completely random value from the set, if any exist.
+   Keys with value zero cannot be distinguished from a nonexistent key.  */
+void *
+ctf_dynset_lookup_any (ctf_dynset_t *hp)
+{
+  struct htab *htab = (struct htab *) hp;
+  void **slot = htab->entries;
+  void **limit = slot + htab_size (htab);
+
+  while (slot < limit
+	 && (*slot == HTAB_EMPTY_ENTRY || *slot == HTAB_DELETED_ENTRY))
+      slot++;
+
+  if (slot < limit)
+    return internal_to_key (*slot);
+  return NULL;
+}
+
 /* ctf_hash, used for fixed-size maps from const char * -> ctf_id_t without
    removal.  This is a straight cast of a hashtab.  */
 
@@ -415,12 +561,12 @@ ctf_hash_insert_type (ctf_hash_t *hp, ctf_file_t *fp, uint32_t type,
 
 /* if the key is already in the hash, override the previous definition with
    this new official definition. If the key is not present, then call
-   ctf_hash_insert_type() and hash it in.  */
+   ctf_hash_insert_type and hash it in.  */
 int
 ctf_hash_define_type (ctf_hash_t *hp, ctf_file_t *fp, uint32_t type,
                       uint32_t name)
 {
-  /* This matches the semantics of ctf_hash_insert_type() in this
+  /* This matches the semantics of ctf_hash_insert_type in this
      implementation anyway.  */
 
   return ctf_hash_insert_type (hp, fp, type, name);
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 3e097795bc6..7d583ab6679 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -34,6 +34,7 @@
 #include <ctype.h>
 #include <elf.h>
 #include <bfd.h>
+#include "hashtab.h"
 
 #ifdef	__cplusplus
 extern "C"
@@ -73,6 +74,7 @@ extern "C"
 
 typedef struct ctf_fixed_hash ctf_hash_t; /* Private to ctf-hash.c.  */
 typedef struct ctf_dynhash ctf_dynhash_t; /* Private to ctf-hash.c.  */
+typedef struct ctf_dynset ctf_dynset_t;   /* Private to ctf-hash.c.  */
 
 typedef struct ctf_strs
 {
@@ -378,6 +380,8 @@ extern int ctf_hash_eq_integer (const void *, const void *);
 extern int ctf_hash_eq_string (const void *, const void *);
 extern int ctf_hash_eq_type_mapping_key (const void *, const void *);
 
+extern int ctf_dynset_eq_string (const void *, const void *);
+
 typedef void (*ctf_hash_free_fun) (void *);
 
 typedef void (*ctf_hash_iter_f) (void *key, void *value, void *arg);
@@ -407,6 +411,15 @@ extern void ctf_dynhash_iter_remove (ctf_dynhash_t *, ctf_hash_iter_remove_f,
 extern void *ctf_dynhash_iter_find (ctf_dynhash_t *, ctf_hash_iter_find_f,
 				    void *);
 
+extern ctf_dynset_t *ctf_dynset_create (htab_hash, htab_eq, ctf_hash_free_fun);
+extern int ctf_dynset_insert (ctf_dynset_t *, void *);
+extern void ctf_dynset_remove (ctf_dynset_t *, const void *);
+extern void ctf_dynset_destroy (ctf_dynset_t *);
+extern void *ctf_dynset_lookup (ctf_dynset_t *, const void *);
+extern int ctf_dynset_exists (ctf_dynset_t *, const void *key,
+			      const void **orig_key);
+extern void *ctf_dynset_lookup_any (ctf_dynset_t *);
+
 #define	ctf_list_prev(elem)	((void *)(((ctf_list_t *)(elem))->l_prev))
 #define	ctf_list_next(elem)	((void *)(((ctf_list_t *)(elem))->l_next))
 
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
index ad74b39f539..f9c1b2adbfb 100644
--- a/libctf/ctf-inlines.h
+++ b/libctf/ctf-inlines.h
@@ -38,6 +38,12 @@ ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
   return ctf_dynhash_insert (h, (void *) k, (void *) v);
 }
 
+static inline int
+ctf_dynset_cinsert (ctf_dynset_t *h, const void *k)
+{
+  return ctf_dynset_insert (h, (void *) k);
+}
+
 #ifdef	__cplusplus
 }
 #endif
-- 
2.27.0.247.g3dff7de930

[PATCH 24/59] libctf: move existing inlines into ctf-inlines.h

[Nick Alcock]

Just housekeeping.

libctf/
	* ctf-impl.h (ctf_get_ctt_size): Move definition from here...
	* ctf-inlines.h (ctf_get_ctt_size): ... to here.
---
 libctf/ctf-impl.h    | 8 --------
 libctf/ctf-inlines.h | 9 +++++++++
 2 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 7d583ab6679..81a3c62dcfc 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -352,14 +352,6 @@ struct ctf_archive_internal
 #define LCTF_VBYTES(fp, kind, size, vlen) \
   ((fp)->ctf_fileops->ctfo_get_vbytes(kind, size, vlen))
 
-static inline ssize_t ctf_get_ctt_size (const ctf_file_t *fp,
-					const ctf_type_t *tp,
-					ssize_t *sizep,
-					ssize_t *incrementp)
-{
-  return (fp->ctf_fileops->ctfo_get_ctt_size (fp, tp, sizep, incrementp));
-}
-
 #define LCTF_CHILD	0x0001	/* CTF container is a child */
 #define LCTF_RDWR	0x0002	/* CTF container is writable */
 #define LCTF_DIRTY	0x0004	/* CTF container has been modified */
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
index f9c1b2adbfb..4bec97b11b3 100644
--- a/libctf/ctf-inlines.h
+++ b/libctf/ctf-inlines.h
@@ -31,6 +31,15 @@ extern "C"
 #error "ctf-inlines.h" should not be included directly: include "ctf-impl.h".
 #endif
 
+static inline ssize_t
+ctf_get_ctt_size (const ctf_file_t *fp,
+		  const ctf_type_t *tp,
+		  ssize_t *sizep,
+		  ssize_t *incrementp)
+{
+  return (fp->ctf_fileops->ctfo_get_ctt_size (fp, tp, sizep, incrementp));
+}
+
 
 static inline int
 ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
-- 
2.27.0.247.g3dff7de930

[PATCH 25/59] libctf: add ctf_forwardable_kind

[Nick Alcock]

The internals of the deduplicator want to know if something is a type
that can have a forward to it fairly often, often enough that inlining
it brings a noticeable performance gain.  Convert the one place in
libctf that can already benefit, even though it doesn't bring any sort
of performance gain there.

libctf/
	* ctf-inlines.h (ctf_forwardable_kind): New.
	* ctf-create.c (ctf_add_forward): Use it.
---
 libctf/ctf-create.c  | 2 +-
 libctf/ctf-inlines.h | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index c13b83d4d3b..d50367d6de3 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -1241,7 +1241,7 @@ ctf_add_forward (ctf_file_t *fp, uint32_t flag, const char *name,
   ctf_dtdef_t *dtd;
   ctf_id_t type = 0;
 
-  if (kind != CTF_K_STRUCT && kind != CTF_K_UNION && kind != CTF_K_ENUM)
+  if (!ctf_forwardable_kind (kind))
     return (ctf_set_errno (fp, ECTF_NOTSUE));
 
   /* If the type is already defined or exists as a forward tag, just
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
index 4bec97b11b3..3b912bd9a25 100644
--- a/libctf/ctf-inlines.h
+++ b/libctf/ctf-inlines.h
@@ -40,6 +40,12 @@ ctf_get_ctt_size (const ctf_file_t *fp,
   return (fp->ctf_fileops->ctfo_get_ctt_size (fp, tp, sizep, incrementp));
 }
 
+static inline int
+ctf_forwardable_kind (int kind)
+{
+  return (kind == CTF_K_STRUCT || kind == CTF_K_UNION || kind == CTF_K_ENUM);
+}
+
 
 static inline int
 ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
-- 
2.27.0.247.g3dff7de930

[PATCH 26/59] libctf: add ctf_ref

[Nick Alcock]

This allows you to bump the refcount on a ctf_file_t, so that you can
smuggle it out of iterators which open and close the ctf_file_t for you
around the loop body (like ctf_archive_iter).

You still can't use this to preserve a ctf_file_t for longer than the
lifetime of its containing entity (e.g. ctf_archive).

include/
	* ctf-api.h (ctf_ref): New.
libctf/
	* libctf.ver (ctf_ref): New.
	* ctf-open.c (ctf_ref): Implement it.
---
 include/ctf-api.h |  1 +
 libctf/ctf-open.c | 11 +++++++++++
 libctf/libctf.ver |  1 +
 3 files changed, 13 insertions(+)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index fb797a3346d..faa7b727fb8 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -285,6 +285,7 @@ extern ctf_file_t *ctf_simple_open (const char *, size_t, const char *, size_t,
 				   size_t, const char *, size_t, int *);
 extern ctf_file_t *ctf_bufopen (const ctf_sect_t *, const ctf_sect_t *,
 				const ctf_sect_t *, int *);
+extern void ctf_ref (ctf_file_t *);
 extern void ctf_file_close (ctf_file_t *);
 
 extern int ctf_arc_write (const char *, ctf_file_t **, size_t,
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index 05672f36498..b7846bd0014 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1620,6 +1620,17 @@ bad:
   return NULL;
 }
 
+/* Bump the refcount on the specified CTF container, to allow export of
+   ctf_file_t's from iterators that open and close the ctf_file_t around the
+   loop.  (This does not extend their lifetime beyond that of the ctf_archive_t
+   in which they are contained.)  */
+
+void
+ctf_ref (ctf_file_t *fp)
+{
+  fp->ctf_refcnt++;
+}
+
 /* Close the specified CTF container and free associated data structures.  Note
    that ctf_file_close() is a reference counted operation: if the specified file
    is the parent of other active containers, its reference count will be greater
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 375dee8fc77..93616d83691 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -24,6 +24,7 @@ LIBCTF_1.0 {
 	ctf_simple_open;
 	ctf_create;
 	ctf_close;
+	ctf_ref;
 	ctf_file_close;
 
 	ctf_cuname;
-- 
2.27.0.247.g3dff7de930

[PATCH 27/59] libctf, next: introduce new class of easier-to-use iterators

[Nick Alcock]

The libctf machinery currently only provides one way to iterate over its
data structures: ctf_*_iter functions that take a callback and an arg
and repeatedly call it.

This *works*, but if you are doing a lot of iteration it is really quite
inconvenient: you have to package up your local variables into
structures over and over again and spawn lots of little functions even
if it would be clearer in a single run of code.  Look at ctf-string.c
for an extreme example of how unreadable this can get, with
three-line-long functions proliferating wildly.

The deduplicator takes this to the Nth level. It iterates over a whole
bunch of things: if we'd had to use _iter-class iterators for all of
them there would be twenty additional functions in the deduplicator
alone, for no other reason than that the iterator API requires it.

Let's do something better. strtok_r gives us half the design: generators
in a number of other languages give us the other half.

The *_next API allows you to iterate over CTF-like entities in a single
function using a normal while loop. e.g. here we are iterating over all
the types in a dict:

ctf_next_t *i = NULL;
int *hidden;
ctf_id_t id;

while ((id = ctf_type_next (fp, &i, &hidden, 1)) != CTF_ERR)
  {
    /* do something with 'hidden' and 'id' */
  }
if (ctf_errno (fp) != ECTF_NEXT_END)
    /* iteration error */

Here we are walking through the members of a struct with CTF ID
'struct_type':

ctf_next_t *i = NULL;
ssize_t offset;
const char *name;
ctf_id_t membtype;

while ((offset = ctf_member_next (fp, struct_type, &i, &name,
                                  &membtype)) >= 0
  {
    /* do something with offset, name, and membtype */
  }
if (ctf_errno (fp) != ECTF_NEXT_END)
    /* iteration error */

Like every other while loop, this means you have access to all the local
variables outside the loop while inside it, with no need to tiresomely
package things up in structures, move the body of the loop into a
separate function, etc, as you would with an iterator taking a callback.

ctf_*_next allocates 'i' for you on first entry (when it must be NULL),
and frees and NULLs it and returns a _next-dependent flag value when the
iteration is over: the fp errno is set to ECTF_NEXT_END when the
iteartion ends normally.  If you want to exit early, call
ctf_next_destroy on the iterator.  You can copy iterators using
ctf_next_copy, which copies their current iteration position so you can
remember loop positions and go back to them later (or ctf_next_destroy
them if you don't need them after all).

Each _next function returns an always-likely-to-be-useful property of
the thing being iterated over, and takes pointers to parameters for the
others: with very few exceptions all those parameters can be NULLs if
you're not interested in them, so e.g. you can iterate over only the
offsets of members of a structure this way:

while ((offset = ctf_member_next (fp, struct_id, &i, NULL, NULL)) >= 0)

If you pass an iterator in use by one iteration function to another one,
you get the new error ECTF_NEXT_WRONGFUN back; if you try to change
ctf_file_t in mid-iteration, you get ECTF_NEXT_WRONGFP back.

Internally the ctf_next_t remembers the iteration function in use,
various sizes and increments useful for almost all iterations, then
uses unions to overlap the actual entities being iterated over to keep
ctf_next_t size down.

Iterators available in the public API so far (all tested in actual use
in the deduplicator):

/* Iterate over the members of a STRUCT or UNION, returning each member's
   offset and optionally name and member type in turn.  On end-of-iteration,
   returns -1.  */
ssize_t
ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
                 const char **name, ctf_id_t *membtype);

/* Iterate over the members of an enum TYPE, returning each enumerand's
   NAME or NULL at end of iteration or error, and optionally passing
   back the enumerand's integer VALue.  */
const char *
ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
              int *val);

/* Iterate over every type in the given CTF container (not including
   parents), optionally including non-user-visible types, returning
   each type ID and optionally the hidden flag in turn. Returns CTF_ERR
   on end of iteration or error.  */
ctf_id_t
ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag,
               int want_hidden);

/* Iterate over every variable in the given CTF container, in arbitrary
   order, returning the name and type of each variable in turn.  The
   NAME argument is not optional.  Returns CTF_ERR on end of iteration
   or error.  */
ctf_id_t
ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name);

/* Iterate over all CTF files in an archive, returning each dict in turn as a
   ctf_file_t, and NULL on error or end of iteration.  It is the caller's
   responsibility to close it.  Parent dicts may be skipped.  Regardless of
   whether they are skipped or not, the caller must ctf_import the parent if
   need be.  */
ctf_file_t *
ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it,
                  const char **name, int skip_parent, int *errp);

ctf_label_next is prototyped but not implemented yet.

include/
	* ctf-api.h (ECTF_NEXT_END): New error.
	(ECTF_NEXT_WRONGFUN): Likewise.
	(ECTF_NEXT_WRONGFP): Likewise.
	(ECTF_NERR): Adjust.
	(ctf_next_t): New.
	(ctf_next_create): New prototype.
	(ctf_next_destroy): Likewise.
	(ctf_next_copy): Likewise.
	(ctf_member_next): Likewise.
	(ctf_enum_next): Likewise.
	(ctf_type_next): Likewise.
	(ctf_label_next): Likewise.
	(ctf_variable_next): Likewise.

libctf/
	* ctf-impl.h (ctf_next): New.
	(ctf_get_dict): New prototype.
	* ctf-lookup.c (ctf_get_dict): New, split out of...
	(ctf_lookup_by_id): ... here.
	* ctf-util.c (ctf_next_create): New.
	(ctf_next_destroy): New.
	(ctf_next_copy): New.
	* ctf-types.c (includes): Add <assert.h>.
	(ctf_member_next): New.
	(ctf_enum_next): New.
	(ctf_type_iter): Document the lack of iteration over parent
	types.
	(ctf_type_next): New.
	(ctf_variable_next): New.
	* ctf-archive.c (ctf_archive_next): New.
	* libctf.ver: Add new public functions.
---
 include/ctf-api.h    |  33 +++-
 libctf/ctf-archive.c | 106 +++++++++++++
 libctf/ctf-impl.h    |  32 ++++
 libctf/ctf-lookup.c  |  19 ++-
 libctf/ctf-types.c   | 352 ++++++++++++++++++++++++++++++++++++++++++-
 libctf/ctf-util.c    |  29 ++++
 libctf/libctf.ver    |   9 ++
 7 files changed, 570 insertions(+), 10 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index faa7b727fb8..47a1f732f0e 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -204,10 +204,13 @@ enum
    ECTF_DUMPSECTCHANGED, /* Section changed in middle of dump.  */
    ECTF_NOTYET,		/* Feature not yet implemented.  */
    ECTF_INTERNAL,	/* Internal error in link.  */
-   ECTF_NONREPRESENTABLE /* Type not representable in CTF.  */
+   ECTF_NONREPRESENTABLE, /* Type not representable in CTF.  */
+   ECTF_NEXT_END,	/* End of iteration.  */
+   ECTF_NEXT_WRONGFUN,	/* Wrong iteration function called.  */
+   ECTF_NEXT_WRONGFP	/* Iteration entity changed in mid-iterate.  */
   };
 
-#define ECTF_NERR (ECTF_NONREPRESENTABLE - ECTF_BASE + 1)	/* Count of CTF errors.  */
+#define ECTF_NERR (ECTF_NEXT_WRONGFP - ECTF_BASE + 1)	/* Count of CTF errors.  */
 
 /* The CTF data model is inferred to be the caller's data model or the data
    model of the given object, unless ctf_setmodel() is explicitly called.  */
@@ -227,8 +230,9 @@ enum
 #define	CTF_ADD_NONROOT	0	/* Type only visible in nested scope.  */
 #define	CTF_ADD_ROOT	1	/* Type visible at top-level scope.  */
 
-/* These typedefs are used to define the signature for callback functions
-   that can be used with the iteration and visit functions below.  */
+/* These typedefs are used to define the signature for callback functions that
+   can be used with the iteration and visit functions below.  There is also a
+   family of iteration functions that do not require callbacks.  */
 
 typedef int ctf_visit_f (const char *name, ctf_id_t type, unsigned long offset,
 			 int depth, void *arg);
@@ -248,6 +252,15 @@ typedef char *ctf_dump_decorate_f (ctf_sect_names_t sect,
 
 typedef struct ctf_dump_state ctf_dump_state_t;
 
+/* Iteration state for the _next() functions, and allocators/copiers/freers for
+   it.  (None of these are needed for the simple case of iterating to the end:
+   the _next() function allocate and free the iterators for you.)  */
+
+typedef struct ctf_next ctf_next_t;
+extern ctf_next_t *ctf_next_create (void);
+extern void ctf_next_destroy (ctf_next_t *);
+extern ctf_next_t *ctf_next_copy (ctf_next_t *);
+
 /* Opening.  These mostly return an abstraction over both CTF files and CTF
    archives: so they can be used to open both.  CTF files will appear to be an
    archive with one member named '.ctf'.  The low-level functions
@@ -353,13 +366,25 @@ extern int ctf_label_info (ctf_file_t *, const char *, ctf_lblinfo_t *);
 
 extern int ctf_member_count (ctf_file_t *, ctf_id_t);
 extern int ctf_member_iter (ctf_file_t *, ctf_id_t, ctf_member_f *, void *);
+extern ssize_t ctf_member_next (ctf_file_t *, ctf_id_t, ctf_next_t **,
+				const char **name, ctf_id_t *membtype);
 extern int ctf_enum_iter (ctf_file_t *, ctf_id_t, ctf_enum_f *, void *);
+extern const char *ctf_enum_next (ctf_file_t *, ctf_id_t, ctf_next_t **,
+				  int *);
 extern int ctf_type_iter (ctf_file_t *, ctf_type_f *, void *);
 extern int ctf_type_iter_all (ctf_file_t *, ctf_type_all_f *, void *);
+extern ctf_id_t ctf_type_next (ctf_file_t *, ctf_next_t **,
+			       int *flag, int want_hidden);
 extern int ctf_label_iter (ctf_file_t *, ctf_label_f *, void *);
+extern int ctf_label_next (ctf_file_t *, ctf_next_t **, const char **); /* TBD */
 extern int ctf_variable_iter (ctf_file_t *, ctf_variable_f *, void *);
+extern ctf_id_t ctf_variable_next (ctf_file_t *, ctf_next_t **,
+				   const char **);
 extern int ctf_archive_iter (const ctf_archive_t *, ctf_archive_member_f *,
 			     void *);
+extern ctf_file_t *ctf_archive_next (const ctf_archive_t *, ctf_next_t **,
+				     const char **, int skip_parent, int *errp);
+
 /* This function alone does not currently operate on CTF files masquerading
    as archives, and returns -EINVAL: the raw data is no longer available.  It is
    expected to be used only by archiving tools, in any case, which have no need
diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 1cfe40f6e3a..0d824356739 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -739,6 +739,112 @@ ctf_archive_iter (const ctf_archive_t *arc, ctf_archive_member_f *func,
   return func (arc->ctfi_file, _CTF_SECTION, data);
 }
 
+/* Iterate over all CTF files in an archive, returning each dict in turn as a
+   ctf_file_t, and NULL on error or end of iteration.  It is the caller's
+   responsibility to close it.  Parent dicts may be skipped.  Regardless of
+   whether they are skipped or not, the caller must ctf_import the parent if
+   need be.
+
+   We identify parents by name rather than by flag value: for now, with the
+   linker only emitting parents named _CTF_SECTION, this works well enough.  */
+
+ctf_file_t *
+ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it, const char **name,
+		  int skip_parent, int *errp)
+{
+  ctf_file_t *f;
+  ctf_next_t *i = *it;
+  struct ctf_archive *arc;
+  struct ctf_archive_modent *modent;
+  const char *nametbl;
+  const char *name_;
+
+  if (!i)
+    {
+      if ((i = ctf_next_create()) == NULL)
+	{
+	  if (errp)
+	    *errp = ENOMEM;
+	  return NULL;
+	}
+      i->cu.ctn_arc = wrapper;
+      i->ctn_iter_fun = (void (*) (void)) ctf_archive_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_archive_next != i->ctn_iter_fun)
+    {
+      if (errp)
+	*errp = ECTF_NEXT_WRONGFUN;
+      return NULL;
+    }
+
+  if (wrapper != i->cu.ctn_arc)
+    {
+      if (errp)
+	*errp = ECTF_NEXT_WRONGFP;
+      return NULL;
+    }
+
+  /* Iteration is made a bit more complex by the need to handle ctf_file_t's
+     transparently wrapped in a single-member archive.  These are parents: if
+     skip_parent is on, they are skipped and the iterator terminates
+     immediately.  */
+
+  if (!wrapper->ctfi_is_archive && i->ctn_n == 0)
+    {
+      i->ctn_n++;
+      if (!skip_parent)
+	{
+	  wrapper->ctfi_file->ctf_refcnt++;
+	  return wrapper->ctfi_file;
+	}
+    }
+
+  arc = wrapper->ctfi_archive;
+
+  /* The loop keeps going when skip_parent is on as long as the member we find
+     is the parent (i.e. at most two iterations, but possibly an early return if
+     *all* we have is a parent).  */
+
+  const ctf_sect_t *symsect;
+  const ctf_sect_t *strsect;
+
+  do
+    {
+      if ((!wrapper->ctfi_is_archive) || (i->ctn_n >= le64toh (arc->ctfa_nfiles)))
+	{
+	  ctf_next_destroy (i);
+	  *it = NULL;
+	  if (errp)
+	    *errp = ECTF_NEXT_END;
+	  return NULL;
+	}
+
+      symsect = &wrapper->ctfi_symsect;
+      strsect = &wrapper->ctfi_strsect;
+
+      if (symsect->cts_name == NULL)
+	symsect = NULL;
+      if (strsect->cts_name == NULL)
+	strsect = NULL;
+
+      modent = (ctf_archive_modent_t *) ((char *) arc
+					 + sizeof (struct ctf_archive));
+      nametbl = (((const char *) arc) + le64toh (arc->ctfa_names));
+
+      name_ = &nametbl[le64toh (modent[i->ctn_n++].name_offset)];
+    } while (skip_parent && strcmp (name_, _CTF_SECTION) == 0);
+
+  if (name)
+    *name = name_;
+
+  f = ctf_arc_open_by_name_internal (arc, symsect, strsect,
+				     name_, errp);
+  f->ctf_archive = (ctf_archive_t *) wrapper;
+  return f;
+}
+
 #ifdef HAVE_MMAP
 /* Map the header in.  Only used on new, empty files.  */
 static void *arc_mmap_header (int fd, size_t headersz)
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 81a3c62dcfc..b18ba946524 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -326,6 +326,36 @@ struct ctf_archive_internal
   void (*ctfi_bfd_close) (struct ctf_archive_internal *);
 };
 
+/* Iterator state for the *_next() functions.  */
+
+struct ctf_next
+{
+  void (*ctn_iter_fun) (void);
+  ctf_id_t ctn_type;
+  ssize_t ctn_size;
+  ssize_t ctn_increment;
+  uint32_t ctn_n;
+  /* We can save space on this side of things by noting that a container is
+     either dynamic or not, as a whole, and a given iterator can only iterate
+     over one kind of thing at once: so we can overlap the DTD and non-DTD
+     members, and the structure, variable and enum members, etc.  */
+  union
+  {
+    const ctf_member_t *ctn_mp;
+    const ctf_lmember_t *ctn_lmp;
+    const ctf_dmdef_t *ctn_dmd;
+    const ctf_enum_t *ctn_en;
+    const ctf_dvdef_t *ctn_dvd;
+  } u;
+  /* This union is of various sorts of container we can iterate over:
+     currently dictionaries and archives.  */
+  union
+  {
+    const ctf_file_t *ctn_fp;
+    const ctf_archive_t *ctn_arc;
+  } cu;
+};
+
 /* Return x rounded up to an alignment boundary.
    eg, P2ROUNDUP(0x1234, 0x100) == 0x1300 (0x13*align)
    eg, P2ROUNDUP(0x5600, 0x100) == 0x5600 (0x56*align)  */
@@ -362,6 +392,8 @@ extern ctf_id_t ctf_lookup_by_rawname (ctf_file_t *, int, const char *);
 extern ctf_id_t ctf_lookup_by_rawhash (ctf_file_t *, ctf_names_t *, const char *);
 extern void ctf_set_ctl_hashes (ctf_file_t *);
 
+extern ctf_file_t *ctf_get_dict (ctf_file_t *fp, ctf_id_t type);
+
 typedef unsigned int (*ctf_hash_fun) (const void *ptr);
 extern unsigned int ctf_hash_integer (const void *ptr);
 extern unsigned int ctf_hash_string (const void *ptr);
diff --git a/libctf/ctf-lookup.c b/libctf/ctf-lookup.c
index d95e53706c1..8daab632dca 100644
--- a/libctf/ctf-lookup.c
+++ b/libctf/ctf-lookup.c
@@ -305,6 +305,18 @@ ctf_lookup_by_symbol (ctf_file_t *fp, unsigned long symidx)
   return type;
 }
 
+/* Return the native dict of a given type: if called on a child and the
+   type is in the parent, return the parent.  Needed if you plan to access
+   the type directly, without using the API.  */
+ctf_file_t *
+ctf_get_dict (ctf_file_t *fp, ctf_id_t type)
+{
+    if ((fp->ctf_flags & LCTF_CHILD) && LCTF_TYPE_ISPARENT (fp, type))
+      return fp->ctf_parent;
+
+    return fp;
+}
+
 /* Return the pointer to the internal CTF type data corresponding to the
    given type ID.  If the ID is invalid, the function returns NULL.
    This function is not exported outside of the library.  */
@@ -312,17 +324,16 @@ ctf_lookup_by_symbol (ctf_file_t *fp, unsigned long symidx)
 const ctf_type_t *
 ctf_lookup_by_id (ctf_file_t **fpp, ctf_id_t type)
 {
-  ctf_file_t *fp = *fpp;	/* Caller passes in starting CTF container.  */
+  ctf_file_t *fp = *fpp;	/* Caller passes in starting CTF dict.  */
   ctf_id_t idx;
 
-  if ((fp->ctf_flags & LCTF_CHILD) && LCTF_TYPE_ISPARENT (fp, type)
-      && (fp = fp->ctf_parent) == NULL)
+  if ((fp = ctf_get_dict (fp, type)) == NULL)
     {
       (void) ctf_set_errno (*fpp, ECTF_NOPARENT);
       return NULL;
     }
 
-  /* If this container is writable, check for a dynamic type.  */
+  /* If this dict is writable, check for a dynamic type.  */
 
   if (fp->ctf_flags & LCTF_RDWR)
     {
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index 93967c6aa8d..d7ed0e31dca 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -18,6 +18,7 @@
    <http://www.gnu.org/licenses/>.  */
 
 #include <ctf-impl.h>
+#include <assert.h>
 #include <string.h>
 
 /* Determine whether a type is a parent or a child.  */
@@ -103,6 +104,125 @@ ctf_member_iter (ctf_file_t *fp, ctf_id_t type, ctf_member_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over the members of a STRUCT or UNION, returning each member's
+   offset and optionally name and member type in turn.  On end-of-iteration,
+   returns -1.  */
+
+ssize_t
+ctf_member_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
+		 const char **name, ctf_id_t *membtype)
+{
+  ctf_file_t *ofp = fp;
+  uint32_t kind;
+  ssize_t offset;
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      const ctf_type_t *tp;
+      ctf_dtdef_t *dtd;
+
+      if ((type = ctf_type_resolve (fp, type)) == CTF_ERR)
+	return -1;			/* errno is set for us.  */
+
+      if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+	return -1;			/* errno is set for us.  */
+
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (ofp, ENOMEM);
+      i->cu.ctn_fp = ofp;
+
+      (void) ctf_get_ctt_size (fp, tp, &i->ctn_size,
+			       &i->ctn_increment);
+      kind = LCTF_INFO_KIND (fp, tp->ctt_info);
+
+      if (kind != CTF_K_STRUCT && kind != CTF_K_UNION)
+	{
+	  ctf_next_destroy (i);
+	  return (ctf_set_errno (ofp, ECTF_NOTSOU));
+	}
+
+      dtd = ctf_dynamic_type (fp, type);
+      i->ctn_iter_fun = (void (*) (void)) ctf_member_next;
+
+      /* We depend below on the RDWR state indicating whether the DTD-related
+	 fields or the DMD-related fields have been initialized.  */
+
+      assert ((dtd && (fp->ctf_flags & LCTF_RDWR))
+	      || (!dtd && (!(fp->ctf_flags & LCTF_RDWR))));
+
+      if (dtd == NULL)
+	{
+	  i->ctn_n = LCTF_INFO_VLEN (fp, tp->ctt_info);
+
+	  if (i->ctn_size < CTF_LSTRUCT_THRESH)
+	    i->u.ctn_mp = (const ctf_member_t *) ((uintptr_t) tp +
+						  i->ctn_increment);
+	  else
+	    i->u.ctn_lmp = (const ctf_lmember_t *) ((uintptr_t) tp +
+						    i->ctn_increment);
+	}
+      else
+	i->u.ctn_dmd = ctf_list_next (&dtd->dtd_u.dtu_members);
+
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_member_next != i->ctn_iter_fun)
+    return (ctf_set_errno (ofp, ECTF_NEXT_WRONGFUN));
+
+  if (ofp != i->cu.ctn_fp)
+    return (ctf_set_errno (ofp, ECTF_NEXT_WRONGFP));
+
+  /* Resolve to the native dict of this type.  */
+  if ((fp = ctf_get_dict (ofp, type)) == NULL)
+    return (ctf_set_errno (ofp, ECTF_NOPARENT));
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n == 0)
+	goto end_iter;
+
+      if (i->ctn_size < CTF_LSTRUCT_THRESH)
+	{
+	  if (name)
+	    *name = ctf_strptr (fp, i->u.ctn_mp->ctm_name);
+	  if (membtype)
+	    *membtype = i->u.ctn_mp->ctm_type;
+	  offset = i->u.ctn_mp->ctm_offset;
+	  i->u.ctn_mp++;
+	}
+      else
+	{
+	  if (name)
+	    *name = ctf_strptr (fp, i->u.ctn_lmp->ctlm_name);
+	  if (membtype)
+	    *membtype = i->u.ctn_lmp->ctlm_type;
+	  offset = (unsigned long) CTF_LMEM_OFFSET (i->u.ctn_lmp);
+	  i->u.ctn_lmp++;
+	}
+      i->ctn_n--;
+    }
+  else
+    {
+      if (i->u.ctn_dmd == NULL)
+	goto end_iter;
+      if (name)
+	*name = i->u.ctn_dmd->dmd_name;
+      if (membtype)
+	*membtype = i->u.ctn_dmd->dmd_type;
+      offset = i->u.ctn_dmd->dmd_offset;
+      i->u.ctn_dmd = ctf_list_next (i->u.ctn_dmd);
+    }
+
+  return offset;
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (ofp, ECTF_NEXT_END);
+}
+
 /* Iterate over the members of an ENUM.  We pass the string name and associated
    integer value of each enum element to the specified callback function.  */
 
@@ -154,8 +274,126 @@ ctf_enum_iter (ctf_file_t *fp, ctf_id_t type, ctf_enum_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over the members of an enum TYPE, returning each enumerand's NAME or
+   NULL at end of iteration or error, and optionally passing back the
+   enumerand's integer VALue.  */
+
+const char *
+ctf_enum_next (ctf_file_t *fp, ctf_id_t type, ctf_next_t **it,
+	       int *val)
+{
+  ctf_file_t *ofp = fp;
+  uint32_t kind;
+  const char *name;
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      const ctf_type_t *tp;
+      ctf_dtdef_t *dtd;
+
+      if ((type = ctf_type_resolve_unsliced (fp, type)) == CTF_ERR)
+	return NULL;			/* errno is set for us.  */
+
+      if ((tp = ctf_lookup_by_id (&fp, type)) == NULL)
+	return NULL;			/* errno is set for us.  */
+
+      if ((i = ctf_next_create ()) == NULL)
+	{
+	  ctf_set_errno (ofp, ENOMEM);
+	  return NULL;
+	}
+      i->cu.ctn_fp = ofp;
+
+      (void) ctf_get_ctt_size (fp, tp, NULL,
+			       &i->ctn_increment);
+      kind = LCTF_INFO_KIND (fp, tp->ctt_info);
+
+      if (kind != CTF_K_ENUM)
+	{
+	  ctf_next_destroy (i);
+	  ctf_set_errno (ofp, ECTF_NOTENUM);
+	  return NULL;
+	}
+
+      dtd = ctf_dynamic_type (fp, type);
+      i->ctn_iter_fun = (void (*) (void)) ctf_enum_next;
+
+      /* We depend below on the RDWR state indicating whether the DTD-related
+	 fields or the DMD-related fields have been initialized.  */
+
+      assert ((dtd && (fp->ctf_flags & LCTF_RDWR))
+	      || (!dtd && (!(fp->ctf_flags & LCTF_RDWR))));
+
+      if (dtd == NULL)
+	{
+	  i->ctn_n = LCTF_INFO_VLEN (fp, tp->ctt_info);
+
+	  i->u.ctn_en = (const ctf_enum_t *) ((uintptr_t) tp +
+					      i->ctn_increment);
+	}
+      else
+	i->u.ctn_dmd = ctf_list_next (&dtd->dtd_u.dtu_members);
+
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_enum_next != i->ctn_iter_fun)
+    {
+      ctf_set_errno (ofp, ECTF_NEXT_WRONGFUN);
+      return NULL;
+    }
+
+  if (ofp != i->cu.ctn_fp)
+    {
+      ctf_set_errno (ofp, ECTF_NEXT_WRONGFP);
+      return NULL;
+    }
+
+  /* Resolve to the native dict of this type.  */
+  if ((fp = ctf_get_dict (ofp, type)) == NULL)
+    {
+      ctf_set_errno (ofp, ECTF_NOPARENT);
+      return NULL;
+    }
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n == 0)
+	goto end_iter;
+
+      name = ctf_strptr (fp, i->u.ctn_en->cte_name);
+      if (val)
+	*val = i->u.ctn_en->cte_value;
+      i->u.ctn_en++;
+      i->ctn_n--;
+    }
+  else
+    {
+      if (i->u.ctn_dmd == NULL)
+	goto end_iter;
+
+      name = i->u.ctn_dmd->dmd_name;
+      if (val)
+	*val = i->u.ctn_dmd->dmd_value;
+      i->u.ctn_dmd = ctf_list_next (i->u.ctn_dmd);
+    }
+
+  return name;
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  ctf_set_errno (ofp, ECTF_NEXT_END);
+  return NULL;
+}
+
 /* Iterate over every root (user-visible) type in the given CTF container.
-   We pass the type ID of each type to the specified callback function.  */
+   We pass the type ID of each type to the specified callback function.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
 
 int
 ctf_type_iter (ctf_file_t *fp, ctf_type_f *func, void *arg)
@@ -175,7 +413,11 @@ ctf_type_iter (ctf_file_t *fp, ctf_type_f *func, void *arg)
 }
 
 /* Iterate over every type in the given CTF container, user-visible or not.
-   We pass the type ID of each type to the specified callback function.  */
+   We pass the type ID of each type to the specified callback function.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
 
 int
 ctf_type_iter_all (ctf_file_t *fp, ctf_type_all_f *func, void *arg)
@@ -195,6 +437,55 @@ ctf_type_iter_all (ctf_file_t *fp, ctf_type_all_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over every type in the given CTF container, optionally including
+   non-user-visible types, returning each type ID and hidden flag in turn.
+   Returns CTF_ERR on end of iteration or error.
+
+   Does not traverse parent types: you have to do that explicitly.  This is by
+   design, to avoid traversing them more than once if traversing many children
+   of a single parent.  */
+
+ctf_id_t
+ctf_type_next (ctf_file_t *fp, ctf_next_t **it, int *flag, int want_hidden)
+{
+  ctf_next_t *i = *it;
+
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
+
+      i->cu.ctn_fp = fp;
+      i->ctn_type = 1;
+      i->ctn_iter_fun = (void (*) (void)) ctf_type_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_type_next != i->ctn_iter_fun)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFUN));
+
+  if (fp != i->cu.ctn_fp)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFP));
+
+  while (i->ctn_type <= fp->ctf_typemax)
+    {
+      const ctf_type_t *tp = LCTF_INDEX_TO_TYPEPTR (fp, i->ctn_type);
+
+      if ((!want_hidden) && (!LCTF_INFO_ISROOT (fp, tp->ctt_info)))
+	{
+	  i->ctn_type++;
+	  continue;
+	}
+
+      if (flag)
+	*flag = LCTF_INFO_ISROOT (fp, tp->ctt_info);
+      return LCTF_INDEX_TO_TYPE (fp, i->ctn_type++, fp->ctf_flags & LCTF_CHILD);
+    }
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (fp, ECTF_NEXT_END);
+}
+
 /* Iterate over every variable in the given CTF container, in arbitrary order.
    We pass the name of each variable to the specified callback function.  */
 
@@ -229,6 +520,63 @@ ctf_variable_iter (ctf_file_t *fp, ctf_variable_f *func, void *arg)
   return 0;
 }
 
+/* Iterate over every variable in the given CTF container, in arbitrary order,
+   returning the name and type of each variable in turn.  The name argument is
+   not optional.  Returns CTF_ERR on end of iteration or error.  */
+
+ctf_id_t
+ctf_variable_next (ctf_file_t *fp, ctf_next_t **it, const char **name)
+{
+  ctf_next_t *i = *it;
+
+  if ((fp->ctf_flags & LCTF_CHILD) && (fp->ctf_parent == NULL))
+    return (ctf_set_errno (fp, ECTF_NOPARENT));
+
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
+
+      i->cu.ctn_fp = fp;
+      i->ctn_iter_fun = (void (*) (void)) ctf_variable_next;
+      if (fp->ctf_flags & LCTF_RDWR)
+	i->u.ctn_dvd = ctf_list_next (&fp->ctf_dvdefs);
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_variable_next != i->ctn_iter_fun)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFUN));
+
+  if (fp != i->cu.ctn_fp)
+    return (ctf_set_errno (fp, ECTF_NEXT_WRONGFP));
+
+  if (!(fp->ctf_flags & LCTF_RDWR))
+    {
+      if (i->ctn_n >= fp->ctf_nvars)
+	goto end_iter;
+
+      *name = ctf_strptr (fp, fp->ctf_vars[i->ctn_n].ctv_name);
+      return fp->ctf_vars[i->ctn_n++].ctv_type;
+    }
+  else
+    {
+      ctf_id_t id;
+
+      if (i->u.ctn_dvd == NULL)
+	goto end_iter;
+
+      *name = i->u.ctn_dvd->dvd_name;
+      id = i->u.ctn_dvd->dvd_type;
+      i->u.ctn_dvd = ctf_list_next (i->u.ctn_dvd);
+      return id;
+    }
+
+ end_iter:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ctf_set_errno (fp, ECTF_NEXT_END);
+}
+
 /* Follow a given type through the graph for TYPEDEF, VOLATILE, CONST, and
    RESTRICT nodes until we reach a "base" type node.  This is useful when
    we want to follow a type ID to a node that has members or a size.  To guard
diff --git a/libctf/ctf-util.c b/libctf/ctf-util.c
index 1f745f533b8..5abea6a70bb 100644
--- a/libctf/ctf-util.c
+++ b/libctf/ctf-util.c
@@ -173,3 +173,32 @@ ctf_set_errno (ctf_file_t * fp, int err)
   fp->ctf_errno = err;
   return CTF_ERR;
 }
+
+/* Create a ctf_next_t.  */
+
+ctf_next_t *
+ctf_next_create (void)
+{
+  return calloc (1, sizeof (struct ctf_next));
+}
+
+/* Destroy a ctf_next_t, for early exit from iterators.  */
+
+void
+ctf_next_destroy (ctf_next_t *i)
+{
+  free (i);
+}
+
+/* Copy a ctf_next_t.  */
+
+ctf_next_t *
+ctf_next_copy (ctf_next_t *i)
+{
+  ctf_next_t *i2;
+
+  if ((i2 = ctf_next_create()) == NULL)
+    return NULL;
+  memcpy (i2, i, sizeof (struct ctf_next));
+  return i2;
+}
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 93616d83691..e99f890e0e1 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -73,6 +73,7 @@ LIBCTF_1.0 {
 	ctf_type_compat;
 
 	ctf_member_info;
+	ctf_member_next;
 	ctf_array_info;
 	ctf_member_count;
 
@@ -87,10 +88,17 @@ LIBCTF_1.0 {
 
 	ctf_member_iter;
 	ctf_enum_iter;
+	ctf_enum_next;
 	ctf_type_iter;
+	ctf_type_next;
 	ctf_type_iter_all;
 	ctf_label_iter;
 	ctf_variable_iter;
+	ctf_variable_next;
+
+	ctf_next_create;
+	ctf_next_destroy;
+	ctf_next_copy;
 
 	ctf_add_array;
 	ctf_add_const;
@@ -138,6 +146,7 @@ LIBCTF_1.0 {
 	ctf_arc_open_by_name_sections;
 	ctf_archive_count;
 	ctf_archive_iter;
+	ctf_archive_next;
 	ctf_archive_raw_iter;
 	ctf_get_arc;
 
-- 
2.27.0.247.g3dff7de930

[PATCH 28/59] libctf, next, hash: add dynhash and dynset _next iteration

[Nick Alcock]

This lets you iterate over dynhashes and dynsets using the _next API.
dynhashes can be iterated over in sorted order, which works by
populating an array of key/value pairs using ctf_dynhash_next itself,
then sorting it with qsort.

Convenience inline functions named ctf_dyn{hash,set}_cnext are also
provided that take (-> return) const keys and values.

libctf/
	* ctf-impl.h (ctf_next_hkv_t): New, kv-pairs passed to
	sorting functions.
	(ctf_next_t) <u.ctn_sorted_hkv>: New, sorted kv-pairs for
	ctf_dynhash_next_sorted.
	<cu.ctn_h>: New, pointer to the dynhash under iteration.
	<cu.ctn_s>: New, pointer to the dynset under iteration.
	(ctf_hash_sort_f): Sorting function passed to...
	(ctf_dynhash_next_sorted): ... this new function.
	(ctf_dynhash_next): New.
	(ctf_dynset_next): New.
	* ctf-inlines.h (ctf_dynhash_cnext_sorted): New.
	(ctf_dynhash_cnext): New.
	(ctf_dynset_cnext): New.
	* ctf-hash.c (ctf_dynhash_next_sorted): New.
	(ctf_dynhash_next): New.
	(ctf_dynset_next): New.
	* ctf-util.c (ctf_next_destroy): Free the u.ctn_sorted_hkv if
	needed.
	(ctf_next_copy): Alloc-and-copy the u.ctn_sorted_hkv if needed.
---
 libctf/ctf-hash.c    | 225 +++++++++++++++++++++++++++++++++++++++++++
 libctf/ctf-impl.h    |  21 +++-
 libctf/ctf-inlines.h |  21 ++++
 libctf/ctf-util.c    |  17 ++++
 4 files changed, 283 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 7eba494a51d..a026ef225da 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -378,6 +378,163 @@ ctf_dynhash_iter_remove (ctf_dynhash_t *hp, ctf_hash_iter_remove_f fun,
   htab_traverse (hp->htab, ctf_hashtab_traverse_remove, &arg);
 }
 
+/* Traverse a dynhash in arbitrary order, in _next iterator form.
+
+   Mutating the dynhash while iterating is not supported (just as it isn't for
+   htab_traverse).
+
+   Note: unusually, this returns zero on success and a *positive* value on
+   error, because it does not take an fp, taking an error pointer would be
+   incredibly clunky, and nearly all error-handling ends up stuffing the result
+   of this into some sort of errno or ctf_errno, which is invariably
+   positive.  So doing this simplifies essentially all callers.  */
+int
+ctf_dynhash_next (ctf_dynhash_t *h, ctf_next_t **it, void **key, void **value)
+{
+  ctf_next_t *i = *it;
+  ctf_helem_t *slot;
+
+  if (!i)
+    {
+      size_t size = htab_size (h->htab);
+
+      /* If the table has too many entries to fit in an ssize_t, just give up.
+	 This might be spurious, but if any type-related hashtable has ever been
+	 nearly as large as that then something very odd is going on.  */
+      if (((ssize_t) size) < 0)
+	return EDOM;
+
+      if ((i = ctf_next_create ()) == NULL)
+	return ENOMEM;
+
+      i->u.ctn_hash_slot = h->htab->entries;
+      i->cu.ctn_h = h;
+      i->ctn_n = 0;
+      i->ctn_size = (ssize_t) size;
+      i->ctn_iter_fun = (void (*) (void)) ctf_dynhash_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_dynhash_next != i->ctn_iter_fun)
+    return ECTF_NEXT_WRONGFUN;
+
+  if (h != i->cu.ctn_h)
+    return ECTF_NEXT_WRONGFP;
+
+  if ((ssize_t) i->ctn_n == i->ctn_size)
+    goto hash_end;
+
+  while ((ssize_t) i->ctn_n < i->ctn_size
+	 && (*i->u.ctn_hash_slot == HTAB_EMPTY_ENTRY
+	     || *i->u.ctn_hash_slot == HTAB_DELETED_ENTRY))
+    {
+      i->u.ctn_hash_slot++;
+      i->ctn_n++;
+    }
+
+  if ((ssize_t) i->ctn_n == i->ctn_size)
+    goto hash_end;
+
+  slot = *i->u.ctn_hash_slot;
+
+  if (key)
+    *key = slot->key;
+  if (value)
+    *value = slot->value;
+
+  i->u.ctn_hash_slot++;
+  i->ctn_n++;
+
+  return 0;
+
+ hash_end:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ECTF_NEXT_END;
+}
+
+/* Traverse a sorted dynhash, in _next iterator form.
+
+   See ctf_dynhash_next for notes on error returns, etc.
+
+   Sort keys before iterating over them using the SORT_FUN and SORT_ARG.
+
+   If SORT_FUN is null, thunks to ctf_dynhash_next.  */
+int
+ctf_dynhash_next_sorted (ctf_dynhash_t *h, ctf_next_t **it, void **key,
+			 void **value, ctf_hash_sort_f sort_fun, void *sort_arg)
+{
+  ctf_next_t *i = *it;
+
+  if (sort_fun == NULL)
+    return ctf_dynhash_next (h, it, key, value);
+
+  if (!i)
+    {
+      size_t els = ctf_dynhash_elements (h);
+      ctf_next_t *accum_i = NULL;
+      void *key, *value;
+      int err;
+      ctf_next_hkv_t *walk;
+
+      if (((ssize_t) els) < 0)
+	return EDOM;
+
+      if ((i = ctf_next_create ()) == NULL)
+	return ENOMEM;
+
+      if ((i->u.ctn_sorted_hkv = calloc (els, sizeof (ctf_next_hkv_t))) == NULL)
+	{
+	  ctf_next_destroy (i);
+	  return ENOMEM;
+	}
+      walk = i->u.ctn_sorted_hkv;
+
+      i->cu.ctn_h = h;
+
+      while ((err = ctf_dynhash_next (h, &accum_i, &key, &value)) == 0)
+	{
+	  walk->hkv_key = key;
+	  walk->hkv_value = value;
+	  walk++;
+	}
+      if (err != ECTF_NEXT_END)
+	{
+	  ctf_next_destroy (i);
+	  return err;
+	}
+
+      if (sort_fun)
+	  ctf_qsort_r (i->u.ctn_sorted_hkv, els, sizeof (ctf_next_hkv_t),
+		       (int (*) (const void *, const void *, void *)) sort_fun,
+		       sort_arg);
+      i->ctn_n = 0;
+      i->ctn_size = (ssize_t) els;
+      i->ctn_iter_fun = (void (*) (void)) ctf_dynhash_next_sorted;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_dynhash_next_sorted != i->ctn_iter_fun)
+    return ECTF_NEXT_WRONGFUN;
+
+  if (h != i->cu.ctn_h)
+    return ECTF_NEXT_WRONGFP;
+
+  if ((ssize_t) i->ctn_n == i->ctn_size)
+    {
+      ctf_next_destroy (i);
+      *it = NULL;
+      return ECTF_NEXT_END;
+    }
+
+  if (key)
+    *key = i->u.ctn_sorted_hkv[i->ctn_n].hkv_key;
+  if (value)
+    *value = i->u.ctn_sorted_hkv[i->ctn_n].hkv_value;
+  i->ctn_n++;
+  return 0;
+}
+
 void
 ctf_dynhash_destroy (ctf_dynhash_t *hp)
 {
@@ -515,6 +672,74 @@ ctf_dynset_lookup_any (ctf_dynset_t *hp)
   return NULL;
 }
 
+/* Traverse a dynset in arbitrary order, in _next iterator form.
+
+   Otherwise, just like ctf_dynhash_next.  */
+int
+ctf_dynset_next (ctf_dynset_t *hp, ctf_next_t **it, void **key)
+{
+  struct htab *htab = (struct htab *) hp;
+  ctf_next_t *i = *it;
+  void *slot;
+
+  if (!i)
+    {
+      size_t size = htab_size (htab);
+
+      /* If the table has too many entries to fit in an ssize_t, just give up.
+	 This might be spurious, but if any type-related hashtable has ever been
+	 nearly as large as that then somthing very odd is going on.  */
+
+      if (((ssize_t) size) < 0)
+	return EDOM;
+
+      if ((i = ctf_next_create ()) == NULL)
+	return ENOMEM;
+
+      i->u.ctn_hash_slot = htab->entries;
+      i->cu.ctn_s = hp;
+      i->ctn_n = 0;
+      i->ctn_size = (ssize_t) size;
+      i->ctn_iter_fun = (void (*) (void)) ctf_dynset_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_dynset_next != i->ctn_iter_fun)
+    return ECTF_NEXT_WRONGFUN;
+
+  if (hp != i->cu.ctn_s)
+    return ECTF_NEXT_WRONGFP;
+
+  if ((ssize_t) i->ctn_n == i->ctn_size)
+    goto set_end;
+
+  while ((ssize_t) i->ctn_n < i->ctn_size
+	 && (*i->u.ctn_hash_slot == HTAB_EMPTY_ENTRY
+	     || *i->u.ctn_hash_slot == HTAB_DELETED_ENTRY))
+    {
+      i->u.ctn_hash_slot++;
+      i->ctn_n++;
+    }
+
+  if ((ssize_t) i->ctn_n == i->ctn_size)
+    goto set_end;
+
+  slot = *i->u.ctn_hash_slot;
+
+  if (key)
+    *key = internal_to_key (slot);
+
+  i->u.ctn_hash_slot++;
+  i->ctn_n++;
+
+  return 0;
+
+ set_end:
+  ctf_next_destroy (i);
+  *it = NULL;
+  return ECTF_NEXT_END;
+}
+
 /* ctf_hash, used for fixed-size maps from const char * -> ctf_id_t without
    removal.  This is a straight cast of a hashtab.  */
 
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index b18ba946524..eea5204c51d 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -328,6 +328,13 @@ struct ctf_archive_internal
 
 /* Iterator state for the *_next() functions.  */
 
+/* A single hash key/value pair.  */
+typedef struct ctf_next_hkv
+{
+  void *hkv_key;
+  void *hkv_value;
+} ctf_next_hkv_t;
+
 struct ctf_next
 {
   void (*ctn_iter_fun) (void);
@@ -346,13 +353,17 @@ struct ctf_next
     const ctf_dmdef_t *ctn_dmd;
     const ctf_enum_t *ctn_en;
     const ctf_dvdef_t *ctn_dvd;
+    ctf_next_hkv_t *ctn_sorted_hkv;
+    void **ctn_hash_slot;
   } u;
   /* This union is of various sorts of container we can iterate over:
-     currently dictionaries and archives.  */
+     currently dictionaries and archives, dynhashes, and dynsets.  */
   union
   {
     const ctf_file_t *ctn_fp;
     const ctf_archive_t *ctn_arc;
+    const ctf_dynhash_t *ctn_h;
+    const ctf_dynset_t *ctn_s;
   } cu;
 };
 
@@ -411,6 +422,8 @@ typedef void (*ctf_hash_free_fun) (void *);
 typedef void (*ctf_hash_iter_f) (void *key, void *value, void *arg);
 typedef int (*ctf_hash_iter_remove_f) (void *key, void *value, void *arg);
 typedef int (*ctf_hash_iter_find_f) (void *key, void *value, void *arg);
+typedef int (*ctf_hash_sort_f) (const ctf_next_hkv_t *, const ctf_next_hkv_t *,
+				void *arg);
 
 extern ctf_hash_t *ctf_hash_create (unsigned long, ctf_hash_fun, ctf_hash_eq_fun);
 extern int ctf_hash_insert_type (ctf_hash_t *, ctf_file_t *, uint32_t, uint32_t);
@@ -434,6 +447,11 @@ extern void ctf_dynhash_iter_remove (ctf_dynhash_t *, ctf_hash_iter_remove_f,
 				     void *);
 extern void *ctf_dynhash_iter_find (ctf_dynhash_t *, ctf_hash_iter_find_f,
 				    void *);
+extern int ctf_dynhash_next (ctf_dynhash_t *, ctf_next_t **,
+			     void **key, void **value);
+extern int ctf_dynhash_next_sorted (ctf_dynhash_t *, ctf_next_t **,
+				    void **key, void **value, ctf_hash_sort_f,
+				    void *);
 
 extern ctf_dynset_t *ctf_dynset_create (htab_hash, htab_eq, ctf_hash_free_fun);
 extern int ctf_dynset_insert (ctf_dynset_t *, void *);
@@ -442,6 +460,7 @@ extern void ctf_dynset_destroy (ctf_dynset_t *);
 extern void *ctf_dynset_lookup (ctf_dynset_t *, const void *);
 extern int ctf_dynset_exists (ctf_dynset_t *, const void *key,
 			      const void **orig_key);
+extern int ctf_dynset_next (ctf_dynset_t *, ctf_next_t **, void **key);
 extern void *ctf_dynset_lookup_any (ctf_dynset_t *);
 
 #define	ctf_list_prev(elem)	((void *)(((ctf_list_t *)(elem))->l_prev))
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
index 3b912bd9a25..a35b6cd5a77 100644
--- a/libctf/ctf-inlines.h
+++ b/libctf/ctf-inlines.h
@@ -46,6 +46,21 @@ ctf_forwardable_kind (int kind)
   return (kind == CTF_K_STRUCT || kind == CTF_K_UNION || kind == CTF_K_ENUM);
 }
 
+static inline int
+ctf_dynhash_cnext_sorted (ctf_dynhash_t *h, ctf_next_t **i, const void **key,
+			  const void **value, ctf_hash_sort_f sort_fun,
+			  void *sort_arg)
+{
+  return ctf_dynhash_next_sorted (h, i, (void **) key, (void **) value,
+				  sort_fun, sort_arg);
+}
+
+static inline int
+ctf_dynhash_cnext (ctf_dynhash_t *h, ctf_next_t **it,
+		  const void **key, const void **value)
+{
+  return ctf_dynhash_next (h, it, (void **) key, (void **) value);
+}
 
 static inline int
 ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
@@ -53,6 +68,12 @@ ctf_dynhash_cinsert (ctf_dynhash_t *h, const void *k, const void *v)
   return ctf_dynhash_insert (h, (void *) k, (void *) v);
 }
 
+static inline int
+ctf_dynset_cnext (ctf_dynset_t *h, ctf_next_t **it, const void **key)
+{
+  return ctf_dynset_next (h, it, (void **) key);
+}
+
 static inline int
 ctf_dynset_cinsert (ctf_dynset_t *h, const void *k)
 {
diff --git a/libctf/ctf-util.c b/libctf/ctf-util.c
index 5abea6a70bb..c113dfc5596 100644
--- a/libctf/ctf-util.c
+++ b/libctf/ctf-util.c
@@ -187,6 +187,11 @@ ctf_next_create (void)
 void
 ctf_next_destroy (ctf_next_t *i)
 {
+  if (i == NULL)
+    return;
+
+  if (i->ctn_iter_fun == (void (*) (void)) ctf_dynhash_next_sorted)
+    free (i->u.ctn_sorted_hkv);
   free (i);
 }
 
@@ -200,5 +205,17 @@ ctf_next_copy (ctf_next_t *i)
   if ((i2 = ctf_next_create()) == NULL)
     return NULL;
   memcpy (i2, i, sizeof (struct ctf_next));
+
+  if (i2->ctn_iter_fun == (void (*) (void)) ctf_dynhash_next_sorted)
+    {
+      size_t els = ctf_dynhash_elements ((ctf_dynhash_t *) i->cu.ctn_h);
+      if ((i2->u.ctn_sorted_hkv = calloc (els, sizeof (ctf_next_hkv_t))) == NULL)
+	{
+	  free (i2);
+	  return NULL;
+	}
+      memcpy (i2->u.ctn_sorted_hkv, i->u.ctn_sorted_hkv,
+	      els * sizeof (ctf_next_hkv_t));
+    }
   return i2;
 }
-- 
2.27.0.247.g3dff7de930

[PATCH 29/59] libctf: pass the thunk down properly when wrapping qsort_r

[Nick Alcock]

When wrapping qsort_r on a system like FreeBSD on which the compar
argument comes first, we wrap the passed arg in a thunk so we can pass
down both the caller-supplied comparator function and its argument.  We
should pass the *argument* down to the comparator, not the thunk, which
is basically random nonsense on the stack from the point of view of the
caller of qsort_r.

libctf/
	ctf-decls.h (ctf_qsort_compar_thunk): Fix arg passing.
---
 libctf/ctf-decls.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-decls.h b/libctf/ctf-decls.h
index 51041c592f4..c47a72e722f 100644
--- a/libctf/ctf-decls.h
+++ b/libctf/ctf-decls.h
@@ -46,7 +46,7 @@ ctf_qsort_compar_thunk (void *arg, const void *a, const void *b)
 {
   struct ctf_qsort_arg *qsort_arg = (struct ctf_qsort_arg *) arg;
 
-  return qsort_arg->compar (a, b, arg);
+  return qsort_arg->compar (a, b, qsort_arg->arg);
 }
 
 static inline void
-- 
2.27.0.247.g3dff7de930

[PATCH 30/59] libctf: error out on corrupt CTF with invalid header flags

[Nick Alcock]

If corrupt CTF with invalid header flags is passed in, return the new
error ECTF_FLAGS.

include/
	* ctf-api.h (ECTF_FLAGS): New.
	(ECTF_NERR): Adjust.
	* ctf.h (CTF_F_MAX): New.
libctf/
	* ctf-open.c (ctf_bufopen_internal): Diagnose invalid flags.
---
 include/ctf-api.h | 5 +++--
 include/ctf.h     | 3 ++-
 libctf/ctf-open.c | 3 +++
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 47a1f732f0e..760b1e46dc6 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -207,10 +207,11 @@ enum
    ECTF_NONREPRESENTABLE, /* Type not representable in CTF.  */
    ECTF_NEXT_END,	/* End of iteration.  */
    ECTF_NEXT_WRONGFUN,	/* Wrong iteration function called.  */
-   ECTF_NEXT_WRONGFP	/* Iteration entity changed in mid-iterate.  */
+   ECTF_NEXT_WRONGFP,	/* Iteration entity changed in mid-iterate.  */
+   ECTF_FLAGS		/* CTF header contains flags unknown to libctf.  */
   };
 
-#define ECTF_NERR (ECTF_NEXT_WRONGFP - ECTF_BASE + 1)	/* Count of CTF errors.  */
+#define ECTF_NERR (ECTF_FLAGS - ECTF_BASE + 1)	/* Count of CTF errors.  */
 
 /* The CTF data model is inferred to be the caller's data model or the data
    model of the given object, unless ctf_setmodel() is explicitly called.  */
diff --git a/include/ctf.h b/include/ctf.h
index 168092b650e..f251759afa1 100644
--- a/include/ctf.h
+++ b/include/ctf.h
@@ -199,7 +199,8 @@ typedef struct ctf_header
 #define CTF_VERSION_3 4
 #define CTF_VERSION CTF_VERSION_3 /* Current version.  */
 
-#define CTF_F_COMPRESS	0x1	/* Data buffer is compressed by libctf.  */
+#define CTF_F_COMPRESS	0x1		/* Data buffer is compressed by libctf.  */
+#define CTF_F_MAX	CTF_F_COMPRESS	/* The greatest flag value in use.  */
 
 typedef struct ctf_lblent
 {
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index b7846bd0014..f8eeaab0168 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1384,6 +1384,9 @@ ctf_bufopen_internal (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
   if (pp->ctp_version < CTF_VERSION_3)
     hdrsz = sizeof (ctf_header_v2_t);
 
+  if (_libctf_unlikely_ (pp->ctp_flags > CTF_F_MAX))
+    return (ctf_set_open_errno (errp, ECTF_FLAGS));
+
   if (ctfsect->cts_size < hdrsz)
     return (ctf_set_open_errno (errp, ECTF_NOCTFBUF));
 
-- 
2.27.0.247.g3dff7de930

[PATCH 31/59] libctf, types: ensure the emission of ECTF_NOPARENT

[Nick Alcock]

From: Egeyar Bagcioglu <egeyar.bagcioglu@oracle.com>

ctf_variable_iter was returning a (positive!) error code rather than
setting the error in the passed-in ctf_file_t.

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>

libctf/
	* ctf-types.c (ctf_variable_iter): Fix error return.
---
 libctf/ctf-types.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index d7ed0e31dca..550068250fd 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -495,7 +495,7 @@ ctf_variable_iter (ctf_file_t *fp, ctf_variable_f *func, void *arg)
   int rc;
 
   if ((fp->ctf_flags & LCTF_CHILD) && (fp->ctf_parent == NULL))
-    return ECTF_NOPARENT;
+    return (ctf_set_errno (fp, ECTF_NOPARENT));
 
   if (!(fp->ctf_flags & LCTF_RDWR))
     {
-- 
2.27.0.247.g3dff7de930

[PATCH 32/59] libctf, ld, binutils: add textual error/warning reporting for libctf

[Nick Alcock]

This commit adds a long-missing piece of infrastructure to libctf: the
ability to report errors and warnings using all the power of printf,
rather than being restricted to one errno value.  Internally, libctf
calls ctf_err_warn() to add errors and warnings to a list: a new
iterator ctf_errwarning_next() then consumes this list one by one and
hands it to the caller, which can free it.  New errors and warnings are
added until the list is consumed by the caller or the ctf_file_t is
closed, so you can dump them at intervals.  The caller can of course
choose to print only those warnings it wants.  (I am not sure whether we
want objdump, readelf or ld to print warnings or not: right now I'm
printing them, but maybe we only want to print errors?  This entirely
depends on whether warnings are voluminous things describing e.g. the
inability to emit single types because of name clashes or something.
There are no users of this infrastructure yet, so it's hard to say.)

There is no internationalization here yet, but this at least adds a
place where internationalization can be added, to one of
ctf_errwarning_next or ctf_err_warn.

We also provide a new ctf_assert() function which uses this
infrastructure to provide non-fatal assertion failures while emitting an
assert-like string to the caller: to save space and avoid needlessly
duplicating unchanging strings, the assertion test is inlined but the
print-things-out failure case is not.  All assertions in libctf will be
converted to use this machinery in future commits and propagate
assertion-failure errors up, so that the linker in particular cannot be
killed by libctf assertion failures when it could perfectly well just
print warnings and drop the CTF section.

include/
	* ctf-api.h (ECTF_INTERNAL): Adjust error text.
	(ctf_errwarning_next): New.
libctf/
	* ctf-impl.h (ctf_assert): New.
	(ctf_err_warning_t): Likewise.
	(ctf_file_t) <ctf_errs_warnings>: Likewise.
	(ctf_err_warn): New prototype.
	(ctf_assert_fail_internal): Likewise.
	* ctf-inlines.h (ctf_assert_internal): Likewise.
	* ctf-open.c (ctf_file_close): Free ctf_errs_warnings.
	* ctf-create.c (ctf_serialize): Copy it on serialization.
	* ctf-subr.c (ctf_err_warn): New, add an error/warning.
	(ctf_errwarning_next): New iterator, free and pass back
	errors/warnings in succession.
	* libctf.ver (ctf_errwarning_next): Add.
ld/
	* ldlang.c (lang_ctf_errs_warnings): New, print CTF errors
	and warnings.  Assert when libctf asserts.
	(lang_merge_ctf): Call it.
	(land_write_ctf): Likewise.
binutils/
	* objdump.c (ctf_archive_member): Print CTF errors and warnings.
	* readelf.c (dump_ctf_archive_member): Likewise.
---
 binutils/objdump.c   | 17 ++++++++
 binutils/readelf.c   | 23 ++++++++++-
 include/ctf-api.h    |  8 +++-
 ld/ldlang.c          | 26 +++++++++++++
 libctf/ctf-create.c  |  2 +
 libctf/ctf-impl.h    | 17 ++++++++
 libctf/ctf-inlines.h | 10 +++++
 libctf/ctf-open.c    |  9 +++++
 libctf/ctf-subr.c    | 93 ++++++++++++++++++++++++++++++++++++++++++++
 libctf/libctf.ver    |  1 +
 10 files changed, 203 insertions(+), 3 deletions(-)

diff --git a/binutils/objdump.c b/binutils/objdump.c
index 93508e2a348..8d4b1b6ba94 100644
--- a/binutils/objdump.c
+++ b/binutils/objdump.c
@@ -4081,6 +4081,9 @@ dump_ctf_archive_member (ctf_file_t *ctf, const char *name, void *arg)
 			  "Function objects", "Variables", "Types", "Strings",
 			  ""};
   const char **thing;
+  ctf_next_t *it = NULL;
+  char *errtext;
+  int is_warning;
   size_t i;
 
   /* Only print out the name of non-default-named archive members.
@@ -4117,6 +4120,20 @@ dump_ctf_archive_member (ctf_file_t *ctf, const char *name, void *arg)
 	  break;
 	}
     }
+
+  /* Dump accumulated errors and warnings.  */
+  while ((errtext = ctf_errwarning_next (ctf, &it, &is_warning)) != NULL)
+    {
+      non_fatal (_("%s: `%s'"), is_warning ? _("warning"): _("error"),
+		 errtext);
+      free (errtext);
+    }
+  if (ctf_errno (ctf) != ECTF_NEXT_END)
+    {
+      non_fatal (_("CTF error: cannot get CTF errors: `%s'"),
+		 ctf_errmsg (ctf_errno (ctf)));
+    }
+
   return 0;
 }
 
diff --git a/binutils/readelf.c b/binutils/readelf.c
index 1d7cfbcf031..f02a9aa6cd9 100644
--- a/binutils/readelf.c
+++ b/binutils/readelf.c
@@ -14173,7 +14173,11 @@ dump_ctf_archive_member (ctf_file_t *ctf, const char *name, void *arg)
 			  "Function objects", "Variables", "Types", "Strings",
 			  ""};
   const char **thing;
+  ctf_next_t *it = NULL;
+  char *errtext;
+  int is_warning;
   size_t i;
+  int err = 0;
 
   /* Only print out the name of non-default-named archive members.
      The name .ctf appears everywhere, even for things that aren't
@@ -14206,10 +14210,25 @@ dump_ctf_archive_member (ctf_file_t *ctf, const char *name, void *arg)
 	{
 	  error (_("Iteration failed: %s, %s\n"), *thing,
 		 ctf_errmsg (ctf_errno (ctf)));
-	  return 1;
+	  err = 1;
+	  goto out;
 	}
     }
-  return 0;
+
+ out:
+  /* Dump accumulated errors and warnings.  */
+  while ((errtext = ctf_errwarning_next (ctf, &it, &is_warning)) != NULL)
+    {
+      error (_("%s: `%s'\n"), is_warning ? _("warning"): _("error"),
+	     errtext);
+      free (errtext);
+    }
+  if (ctf_errno (ctf) != ECTF_NEXT_END)
+    {
+      error (_("CTF error: cannot get CTF errors: `%s'\n"),
+	     ctf_errmsg (ctf_errno (ctf)));
+    }
+  return err;
 }
 
 static bfd_boolean
diff --git a/include/ctf-api.h b/include/ctf-api.h
index 760b1e46dc6..e061b7022b6 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -203,7 +203,7 @@ enum
    ECTF_DUMPSECTUNKNOWN, /* Unknown section number in dump.  */
    ECTF_DUMPSECTCHANGED, /* Section changed in middle of dump.  */
    ECTF_NOTYET,		/* Feature not yet implemented.  */
-   ECTF_INTERNAL,	/* Internal error in link.  */
+   ECTF_INTERNAL,	/* Internal error: assertion failure.  */
    ECTF_NONREPRESENTABLE, /* Type not representable in CTF.  */
    ECTF_NEXT_END,	/* End of iteration.  */
    ECTF_NEXT_WRONGFUN,	/* Wrong iteration function called.  */
@@ -396,6 +396,12 @@ extern char *ctf_dump (ctf_file_t *, ctf_dump_state_t **state,
 		       ctf_sect_names_t sect, ctf_dump_decorate_f *,
 		       void *arg);
 
+/* Error-warning reporting: an 'iterator' that returns errors and warnings from
+   the error/warning list, in order of emission.  Errors and warnings are popped
+   after return: the caller must free the returned error-text pointer.  */
+extern char *ctf_errwarning_next (ctf_file_t *, ctf_next_t **,
+				  int *is_warning);
+
 extern ctf_id_t ctf_add_array (ctf_file_t *, uint32_t,
 			       const ctf_arinfo_t *);
 extern ctf_id_t ctf_add_const (ctf_file_t *, uint32_t, ctf_id_t);
diff --git a/ld/ldlang.c b/ld/ldlang.c
index 23e787a3b22..10bdc86f8d7 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -3726,6 +3726,29 @@ ldlang_open_ctf (void)
     ctf_close (errfile->the_ctf);
 }
 
+/* Emit CTF errors and warnings.  */
+static void
+lang_ctf_errs_warnings (ctf_file_t *fp)
+{
+  ctf_next_t *i = NULL;
+  char *text;
+  int is_warning;
+
+  while ((text = ctf_errwarning_next (fp, &i, &is_warning)) != NULL)
+    {
+      einfo (_("%s: `%s'\n"), is_warning ? _("CTF warning"): _("CTF error"),
+	     text);
+      free (text);
+    }
+  if (ctf_errno (fp) != ECTF_NEXT_END)
+    {
+      einfo (_("CTF error: cannot get CTF errors: `%s'\n"),
+	     ctf_errmsg (ctf_errno (fp)));
+    }
+
+  ASSERT (ctf_errno (fp) != ECTF_INTERNAL);
+}
+
 /* Merge together CTF sections.  After this, only the symtab-dependent
    function and data object sections need adjustment.  */
 
@@ -3779,6 +3802,7 @@ lang_merge_ctf (void)
 	  output_sect->flags |= SEC_EXCLUDE;
 	}
     }
+  lang_ctf_errs_warnings (ctf_output);
 }
 
 /* Let the emulation examine the symbol table and strtab to help it optimize the
@@ -3832,6 +3856,8 @@ lang_write_ctf (int late)
 	  output_sect->size = 0;
 	  output_sect->flags |= SEC_EXCLUDE;
 	}
+
+      lang_ctf_errs_warnings (ctf_output);
     }
 
   /* This also closes every CTF input file used in the link.  */
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index d50367d6de3..a538b2d5603 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -534,6 +534,7 @@ ctf_serialize (ctf_file_t *fp)
   nfp->ctf_ptrtab_len = fp->ctf_ptrtab_len;
   nfp->ctf_link_inputs = fp->ctf_link_inputs;
   nfp->ctf_link_outputs = fp->ctf_link_outputs;
+  nfp->ctf_errs_warnings = fp->ctf_errs_warnings;
   nfp->ctf_str_prov_offset = fp->ctf_str_prov_offset;
   nfp->ctf_syn_ext_strtab = fp->ctf_syn_ext_strtab;
   nfp->ctf_link_cu_mapping = fp->ctf_link_cu_mapping;
@@ -556,6 +557,7 @@ ctf_serialize (ctf_file_t *fp)
   fp->ctf_str_atoms = NULL;
   fp->ctf_prov_strtab = NULL;
   memset (&fp->ctf_dtdefs, 0, sizeof (ctf_list_t));
+  memset (&fp->ctf_errs_warnings, 0, sizeof (ctf_list_t));
   fp->ctf_add_processing = NULL;
   fp->ctf_ptrtab = NULL;
   fp->ctf_link_inputs = NULL;
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index eea5204c51d..47a392723e9 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -70,6 +70,10 @@ extern "C"
 
 #endif
 
+#define ctf_assert(fp, expr)						\
+  _libctf_unlikely_ (ctf_assert_internal (fp, __FILE__, __LINE__,	\
+					  #expr, !!(expr)))
+
 /* libctf in-memory state.  */
 
 typedef struct ctf_fixed_hash ctf_hash_t; /* Private to ctf-hash.c.  */
@@ -195,6 +199,13 @@ typedef struct ctf_bundle
   ctf_dtdef_t *ctb_dtd;		/* CTF dynamic type definition (if any).  */
 } ctf_bundle_t;
 
+typedef struct ctf_err_warning
+{
+  ctf_list_t cew_list;		/* List forward/back pointers.  */
+  int cew_is_warning;		/* 1 if warning, 0 if error.  */
+  char *cew_text;		/* Error/warning text.  */
+} ctf_err_warning_t;
+
 /* Atoms associate strings with a list of the CTF items that reference that
    string, so that ctf_update() can instantiate all the strings using the
    ctf_str_atoms and then reassociate them with the real string later.
@@ -297,6 +308,7 @@ struct ctf_file
   unsigned long ctf_snapshots;	  /* ctf_snapshot() plus ctf_update() count.  */
   unsigned long ctf_snapshot_lu;  /* ctf_snapshot() call count at last update.  */
   ctf_archive_t *ctf_archive;	  /* Archive this ctf_file_t came from.  */
+  ctf_list_t ctf_errs_warnings;	  /* CTF errors and warnings.  */
   ctf_dynhash_t *ctf_link_inputs; /* Inputs to this link.  */
   ctf_dynhash_t *ctf_link_outputs; /* Additional outputs from this link.  */
   ctf_dynhash_t *ctf_link_type_mapping; /* Map input types to output types.  */
@@ -543,6 +555,11 @@ _libctf_printflike_ (1, 2)
 extern void ctf_dprintf (const char *, ...);
 extern void libctf_init_debug (void);
 
+_libctf_printflike_ (3, 4)
+extern void ctf_err_warn (ctf_file_t *, int is_warning, const char *, ...);
+extern void ctf_assert_fail_internal (ctf_file_t *, const char *,
+				      size_t, const char *);
+
 extern Elf64_Sym *ctf_sym_to_elf64 (const Elf32_Sym *src, Elf64_Sym *dst);
 extern const char *ctf_lookup_symbol_name (ctf_file_t *fp, unsigned long symidx);
 
diff --git a/libctf/ctf-inlines.h b/libctf/ctf-inlines.h
index a35b6cd5a77..affc9f91374 100644
--- a/libctf/ctf-inlines.h
+++ b/libctf/ctf-inlines.h
@@ -80,6 +80,16 @@ ctf_dynset_cinsert (ctf_dynset_t *h, const void *k)
   return ctf_dynset_insert (h, (void *) k);
 }
 
+static inline int
+ctf_assert_internal (ctf_file_t *fp, const char *file, size_t line,
+		     const char *exprstr, int expr)
+{
+  if (_libctf_unlikely_ (!expr))
+    ctf_assert_fail_internal (fp, file, line, exprstr);
+
+  return expr;
+}
+
 #ifdef	__cplusplus
 }
 #endif
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index f8eeaab0168..24899f08e20 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1644,6 +1644,7 @@ ctf_file_close (ctf_file_t *fp)
 {
   ctf_dtdef_t *dtd, *ntd;
   ctf_dvdef_t *dvd, *nvd;
+  ctf_err_warning_t *err, *nerr;
 
   if (fp == NULL)
     return;		   /* Allow ctf_file_close(NULL) to simplify caller code.  */
@@ -1710,6 +1711,14 @@ ctf_file_close (ctf_file_t *fp)
   ctf_dynhash_destroy (fp->ctf_link_cu_mapping);
   ctf_dynhash_destroy (fp->ctf_add_processing);
 
+  for (err = ctf_list_next (&fp->ctf_errs_warnings); err != NULL; err = nerr)
+    {
+      nerr = ctf_list_next (err);
+      ctf_list_delete (&fp->ctf_errs_warnings, err);
+      free (err->cew_text);
+      free (err);
+    }
+
   free (fp->ctf_sxlate);
   free (fp->ctf_txlate);
   free (fp->ctf_ptrtab);
diff --git a/libctf/ctf-subr.c b/libctf/ctf-subr.c
index a5cde9d6f20..0b49ae9fca8 100644
--- a/libctf/ctf-subr.c
+++ b/libctf/ctf-subr.c
@@ -194,3 +194,96 @@ void ctf_dprintf (const char *format, ...)
       va_end (alist);
     }
 }
+
+/* Errors and warnings.  */
+_libctf_printflike_ (3, 4)
+extern void
+ctf_err_warn (ctf_file_t *fp, int is_warning, const char *format, ...)
+{
+  va_list alist;
+  ctf_err_warning_t *cew;
+
+  /* Don't bother reporting errors here: we can't do much about them if they
+     happen.  If we're so short of memory that a tiny malloc doesn't work, a
+     vfprintf isn't going to work either and the caller will have to rely on the
+     ENOMEM return they'll be getting in short order anyway.  */
+
+  if ((cew = malloc (sizeof (ctf_err_warning_t))) == NULL)
+    return;
+
+  cew->cew_is_warning = is_warning;
+  va_start (alist, format);
+  if (vasprintf (&cew->cew_text, format, alist) < 0)
+    {
+      free (cew);
+      va_end (alist);
+      return;
+    }
+  va_end (alist);
+
+  ctf_dprintf ("%s: %s\n", is_warning ? "error" : "warning", cew->cew_text);
+
+  ctf_list_append (&fp->ctf_errs_warnings, cew);
+}
+
+/* Error-warning reporting: an 'iterator' that returns errors and warnings from
+   the error/warning list, in order of emission.  Errors and warnings are popped
+   after return: the caller must free the returned error-text pointer.  */
+char *
+ctf_errwarning_next (ctf_file_t *fp, ctf_next_t **it, int *is_warning)
+{
+  ctf_next_t *i = *it;
+  char *ret;
+  ctf_err_warning_t *cew;
+
+  if (!i)
+    {
+      if ((i = ctf_next_create ()) == NULL)
+	{
+	  ctf_set_errno (fp, ENOMEM);
+	  return NULL;
+	}
+
+      i->cu.ctn_fp = fp;
+      i->ctn_iter_fun = (void (*) (void)) ctf_errwarning_next;
+      *it = i;
+    }
+
+  if ((void (*) (void)) ctf_errwarning_next != i->ctn_iter_fun)
+    {
+      ctf_set_errno (fp, ECTF_NEXT_WRONGFUN);
+      return NULL;
+    }
+
+  if (fp != i->cu.ctn_fp)
+    {
+      ctf_set_errno (fp, ECTF_NEXT_WRONGFP);
+      return NULL;
+    }
+
+  cew = ctf_list_next (&fp->ctf_errs_warnings);
+
+  if (!cew)
+    {
+      ctf_next_destroy (i);
+      *it = NULL;
+      ctf_set_errno (fp, ECTF_NEXT_END);
+      return NULL;
+    }
+
+  if (is_warning)
+    *is_warning = cew->cew_is_warning;
+  ret = cew->cew_text;
+  ctf_list_delete (&fp->ctf_errs_warnings, cew);
+  free (cew);
+  return ret;
+}
+
+void
+ctf_assert_fail_internal (ctf_file_t *fp, const char *file, size_t line,
+			  const char *exprstr)
+{
+  ctf_err_warn (fp, 0, "%s: %lu: libctf assertion failed: %s", file,
+		(long unsigned int) line, exprstr);
+  ctf_set_errno (fp, ECTF_INTERNAL);
+}
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index e99f890e0e1..f1c9b2bf003 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -154,6 +154,7 @@ LIBCTF_1.0 {
 
 	ctf_setdebug;
 	ctf_getdebug;
+	ctf_errwarning_next;
 
 	/* Not yet part of the stable API.  */
 
-- 
2.27.0.247.g3dff7de930

[PATCH 33/59] libctf, types: enhance ctf_type_aname to print function arg types

[Nick Alcock]

Somehow this never got implemented, which makes debugging any kind of
bug that has to do with argument types fantastically confusing, because
it *looks* like the func type takes no arguments though in fact it does.

This also lets us simplify the dumper slightly (and introduces our first
uses of ctf_assert and ctf_err_warn: there will be many more).

ctf_type_aname dumps function types without including the function
pointer name itself: ctf_dump search-and-replaces it in.  This seems to
give the nicest-looking results for existing users of both, even if it
is a bit fiddly.

libctf/
	* ctf-types.c (ctf_type_aname): Print arg types here...
	* ctf-dump.c (ctf_dump_funcs): ... not here: but do substitute
	in the type name here.
---
 libctf/ctf-dump.c  | 87 ++++++++++++++++++++--------------------------
 libctf/ctf-types.c | 46 +++++++++++++++++++++++-
 2 files changed, 83 insertions(+), 50 deletions(-)

diff --git a/libctf/ctf-dump.c b/libctf/ctf-dump.c
index b8a81bc1ccf..08d79f36d83 100644
--- a/libctf/ctf-dump.c
+++ b/libctf/ctf-dump.c
@@ -397,13 +397,11 @@ ctf_dump_funcs (ctf_file_t *fp, ctf_dump_state_t *state)
   for (i = 0; i < fp->ctf_nsyms; i++)
     {
       char *str;
-      char *bit;
+      char *bit = NULL;
       const char *err;
       const char *sym_name;
       ctf_funcinfo_t fi;
       ctf_id_t type;
-      size_t j;
-      ctf_id_t *args;
 
       if ((type = ctf_func_info (state->cds_fp, i, &fi)) == CTF_ERR)
 	switch (ctf_errno (state->cds_fp))
@@ -418,74 +416,65 @@ ctf_dump_funcs (ctf_file_t *fp, ctf_dump_state_t *state)
 	  case ECTF_NOFUNCDAT:
 	    continue;
 	  }
-      if ((args = calloc (fi.ctc_argc, sizeof (ctf_id_t))) == NULL)
-	return (ctf_set_errno (fp, ENOMEM));
 
-      /* Return type.  */
-      if ((str = ctf_type_aname (state->cds_fp, type)) == NULL)
+      /* Return type and all args.  */
+      if ((bit = ctf_type_aname (state->cds_fp, type)) == NULL)
 	{
 	  err = "look up return type";
 	  goto err;
 	}
 
-      str = str_append (str, " ");
-
-      /* Function name.  */
+      /* Replace in the returned string, dropping in the function name.  */
 
       sym_name = ctf_lookup_symbol_name (fp, i);
-      if (sym_name[0] == '\0')
+      if (sym_name[0] != '\0')
 	{
-	  if (asprintf (&bit, "0x%lx ", (unsigned long) i) < 0)
-	    goto oom;
-	}
-      else
-	{
-	  if (asprintf (&bit, "%s (0x%lx) ", sym_name, (unsigned long) i) < 0)
+	  char *retstar;
+	  char *new_bit;
+	  char *walk;
+
+	  new_bit = malloc (strlen (bit) + 1 + strlen (sym_name));
+	  if (!new_bit)
 	    goto oom;
-	}
-      str = str_append (str, bit);
-      str = str_append (str, " (");
-      free (bit);
 
-      /* Function arguments.  */
+	  /* See ctf_type_aname.  */
+	  retstar = strstr (bit, "(*) (");
+	  if (!ctf_assert (fp, retstar))
+	    goto assert_err;
+	  retstar += 2;			/* After the '*' */
 
-      if (ctf_func_args (state->cds_fp, i, fi.ctc_argc, args) < 0)
-	{
-	  err = "look up argument type";
-	  goto err;
-	}
+	  /* C is not good at search-and-replace.  */
+	  walk = new_bit;
+	  memcpy (walk, bit, retstar - bit);
+	  walk += (retstar - bit);
+	  strcpy (walk, sym_name);
+	  walk += strlen (sym_name);
+	  strcpy (walk, retstar);
 
-      for (j = 0; j < fi.ctc_argc; j++)
-	{
-	  if ((bit = ctf_type_aname (state->cds_fp, args[j])) == NULL)
-	    {
-	      err = "look up argument type name";
-	      goto err;
-	    }
-	  str = str_append (str, bit);
-	  if ((j < fi.ctc_argc - 1) || (fi.ctc_flags & CTF_FUNC_VARARG))
-	    str = str_append (str, ", ");
 	  free (bit);
+	  bit = new_bit;
 	}
 
-      if (fi.ctc_flags & CTF_FUNC_VARARG)
-	str = str_append (str, "...");
-      str = str_append (str, ")");
+      if (asprintf (&str, "Symbol 0x%lx: %s", (unsigned long) i, bit) < 0)
+	goto oom;
+      free (bit);
 
-      free (args);
       ctf_dump_append (state, str);
       continue;
 
+    err:
+      ctf_err_warn (fp, 1, "Cannot %s dumping function type for "
+		    "symbol 0x%li: %s", err, (unsigned long) i,
+		    ctf_errmsg (ctf_errno (state->cds_fp)));
+      free (bit);
+      return -1;		/* errno is set for us.  */
+
     oom:
-      free (args);
-      free (str);
+      free (bit);
       return (ctf_set_errno (fp, errno));
-    err:
-      ctf_dprintf ("Cannot %s dumping function type for symbol 0x%li: %s\n",
-		   err, (unsigned long) i,
-		   ctf_errmsg (ctf_errno (state->cds_fp)));
-      free (args);
-      free (str);
+
+    assert_err:
+      free (bit);
       return -1;		/* errno is set for us.  */
     }
   return 0;
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index 550068250fd..ddcca66a282 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -745,7 +745,51 @@ ctf_type_aname (ctf_file_t *fp, ctf_id_t type)
 	      ctf_decl_sprintf (&cd, "[%u]", cdp->cd_n);
 	      break;
 	    case CTF_K_FUNCTION:
-	      ctf_decl_sprintf (&cd, "()");
+	      {
+		size_t i;
+		ctf_funcinfo_t fi;
+		ctf_id_t *argv = NULL;
+
+		if (ctf_func_type_info (rfp, cdp->cd_type, &fi) < 0)
+		  goto err;		/* errno is set for us.  */
+
+		if ((argv = calloc (fi.ctc_argc, sizeof (ctf_id_t *))) == NULL)
+		  {
+		    ctf_set_errno (rfp, errno);
+		    goto err;
+		  }
+
+		if (ctf_func_type_args (rfp, cdp->cd_type,
+					fi.ctc_argc, argv) < 0)
+		  goto err;		/* errno is set for us.  */
+
+		ctf_decl_sprintf (&cd, "(*) (");
+		for (i = 0; i < fi.ctc_argc; i++)
+		  {
+		    char *arg = ctf_type_aname (rfp, argv[i]);
+
+		    if (arg == NULL)
+		      goto err;		/* errno is set for us.  */
+		    ctf_decl_sprintf (&cd, "%s", arg);
+		    free (arg);
+
+		    if ((i < fi.ctc_argc - 1)
+			|| (fi.ctc_flags & CTF_FUNC_VARARG))
+		      ctf_decl_sprintf (&cd, ", ");
+		  }
+
+		if (fi.ctc_flags & CTF_FUNC_VARARG)
+		  ctf_decl_sprintf (&cd, "...");
+		ctf_decl_sprintf (&cd, ")");
+
+		free (argv);
+		break;
+
+	      err:
+		free (argv);
+		ctf_decl_fini (&cd);
+		return NULL;
+	      }
 	      break;
 	    case CTF_K_STRUCT:
 	    case CTF_K_FORWARD:
-- 
2.27.0.247.g3dff7de930

[PATCH 34/59] libctf, decl: avoid leaks of the formatted string on error

[Nick Alcock]

ctf_decl_sprintf builds up a formatted string in the ctf_decl_t's
cd_buf, but then on error this is hardly ever freed: we assume that
ctf_decl_fini frees it, but it leaks it instead.

Make it free it like any decent ADT should.

libctf/
	* ctf-decl.c (ctf_decl_fini): Free the cd_buf.
	(ctf_decl_buf): Once it escapes, don't try to free it later.
---
 libctf/ctf-decl.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-decl.c b/libctf/ctf-decl.c
index 5dcf60ab08b..faf421e4765 100644
--- a/libctf/ctf-decl.c
+++ b/libctf/ctf-decl.c
@@ -68,6 +68,7 @@ ctf_decl_fini (ctf_decl_t *cd)
 	  free (cdp);
 	}
     }
+  free (cd->cd_buf);
 }
 
 void
@@ -195,5 +196,7 @@ void ctf_decl_sprintf (ctf_decl_t *cd, const char *format, ...)
 
 char *ctf_decl_buf (ctf_decl_t *cd)
 {
-  return cd->cd_buf;
+  char *buf = cd->cd_buf;
+  cd->cd_buf = NULL;
+  return buf;
 }
-- 
2.27.0.247.g3dff7de930

[PATCH 35/59] libctf, dump: migrate towards dumping errors rather than truncation

[Nick Alcock]

If we get an error emitting a single type, variable, or label, right now
we emit the error into the ctf_dprintf stream and propagate the error
all the way up the stack, causing the entire output to be silently
truncated (unless libctf debugging is on).

Instead, emit an error and keep going.  (This makes sense for this use
case: if you're dumping types and a type is corrupted, you want to
know!)

Not all instances of this are fixed in this commit, only ones associated
with type formatting: more fixes will come.

libctf/
	* ctf-dump.c (ctf_dump_format_type): Emit a warning.
	(ctf_dump_label): Swallow errors from ctf_dump_format_type.
	(ctf_dump_objts): Likewise.
	(ctf_dump_var): Likewise.
	(ctf_dump_type): Do not emit a duplicate message.  Move to
	ctf_err_warning, and swallow all errors.
---
 libctf/ctf-dump.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/libctf/ctf-dump.c b/libctf/ctf-dump.c
index 08d79f36d83..55aa496e8b7 100644
--- a/libctf/ctf-dump.c
+++ b/libctf/ctf-dump.c
@@ -169,6 +169,8 @@ ctf_dump_format_type (ctf_file_t *fp, ctf_id_t id, int flag)
  oom:
   ctf_set_errno (fp, errno);
  err:
+  ctf_err_warn (fp, 1, "Cannot format name dumping type 0x%lx: %s", id,
+		ctf_errmsg (ctf_errno (fp)));
   free (buf);
   free (str);
   free (bit);
@@ -318,7 +320,7 @@ ctf_dump_label (const char *name, const ctf_lblinfo_t *info,
 				       CTF_ADD_ROOT)) == NULL)
     {
       free (str);
-      return -1;			/* errno is set for us.  */
+      return 0;				/* Swallow the error.  */
     }
 
   str = str_append (str, typestr);
@@ -375,7 +377,7 @@ ctf_dump_objts (ctf_file_t *fp, ctf_dump_state_t *state)
 					   CTF_ADD_ROOT)) == NULL)
 	{
 	  free (str);
-	  return -1;			/* errno is set for us.  */
+	  return 0;			/* Swallow the error.  */
 	}
 
       str = str_append (str, typestr);
@@ -495,7 +497,7 @@ ctf_dump_var (const char *name, ctf_id_t type, void *arg)
 				       CTF_ADD_ROOT)) == NULL)
     {
       free (str);
-      return -1;			/* errno is set for us.  */
+      return 0;			/* Swallow the error.  */
     }
 
   str = str_append (str, typestr);
@@ -579,10 +581,7 @@ ctf_dump_type (ctf_id_t id, int flag, void *arg)
   size_t len;
 
   if ((str = ctf_dump_format_type (state->cds_fp, id, flag)) == NULL)
-    {
-      err = "format type";
-      goto err;
-    }
+    goto err_nomsg;		/* Error already logged for us.  */
 
   str = str_append (str, "\n");
   if ((ctf_type_visit (state->cds_fp, id, ctf_dump_member, &membstate)) < 0)
@@ -605,10 +604,11 @@ ctf_dump_type (ctf_id_t id, int flag, void *arg)
   return 0;
 
  err:
-  ctf_dprintf ("Cannot %s dumping type 0x%lx: %s\n", err, id,
-	       ctf_errmsg (ctf_errno (state->cds_fp)));
+  ctf_err_warn (state->cds_fp, 1, "Cannot %s dumping type 0x%lx: %s",
+		err, id, ctf_errmsg (ctf_errno (state->cds_fp)));
+ err_nomsg:
   free (str);
-  return -1;				/* errno is set for us.  */
+  return 0;				/* Swallow the error.  */
 }
 
 /* Dump the string table into the cds_items.  */
-- 
2.27.0.247.g3dff7de930

[PATCH 36/59] libctf, dump: fix slice dumping

[Nick Alcock]

Now that we can have slices of anything terminating in an int, we must
dump things accordingly, or slices of typedefs appear as

  c5b: __u8 -> 16c: __u8 -> 78: short unsigned int (size 0x2)

which is unhelpful.  If things *are* printed as slices, the name is
missing:

  a15: [slice 0x8:0x4]-> 16c: __u8 -> 78: short unsigned int (size 0x2)

And struct members give no clue they're a slice at all, which is a shame
since bitfields are the major use of this type kind:

       [0x8] (ID 0xa15) (kind 10) __u8  dst_reg

Fix things so that everything slicelike or integral gets its encoding
printed, and everything with a name gets the name printed:

  a15: __u8  [slice 0x8:0x4] (size 0x1) -> 1ff: __u8 (size 0x1) -> 37: unsigned char [0x0:0x8] (size 0x1)
     [0x0] (ID 0xa15) (kind 10) __u8:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)

Bitfield struct members get a technically redundant but much
easier-to-understand dumping now:

    [0x0] (ID 0x80000005) (kind 6) struct bpf_insn (aligned at 0x1)
        [0x0] (ID 0x222) (kind 10) __u8 code (aligned at 0x1)
        [0x8] (ID 0x1e9e) (kind 10) __u8  dst_reg:4 (aligned at 0x1, format 0x2, offset:bits 0x8:0x4)
        [0xc] (ID 0x1e46) (kind 10) __u8  src_reg:4 (aligned at 0x1, format 0x2, offset:bits 0xc:0x4)
        [0x10] (ID 0xf35) (kind 10) __s16 off (aligned at 0x2)
        [0x20] (ID 0x1718) (kind 10) __s32 imm (aligned at 0x4)

This also fixes one place where a failure to format a type would be
erroneously considered an out-of-memory condition.

libctf/
	* ctf-dump.c (ctf_is_slice): Delete, unnecessary.
	(ctf_dump_format_type): improve slice formatting.  Always print
	the type size, even of slices.
	(ctf_dump_member): Print slices (-> bitfields) differently from
	non-slices.  Failure to format a type is not an OOM.
---
 libctf/ctf-dump.c | 92 +++++++++++++++++++++++++++++------------------
 1 file changed, 57 insertions(+), 35 deletions(-)

diff --git a/libctf/ctf-dump.c b/libctf/ctf-dump.c
index 55aa496e8b7..94d6bc6f018 100644
--- a/libctf/ctf-dump.c
+++ b/libctf/ctf-dump.c
@@ -79,20 +79,6 @@ ctf_dump_free (ctf_dump_state_t *state)
     }
 }
 
-/* Slices need special handling to distinguish them from their referenced
-   type.  */
-
-static int
-ctf_is_slice (ctf_file_t *fp, ctf_id_t id, ctf_encoding_t *enc)
-{
-  int kind = ctf_type_kind (fp, id);
-
-  return (((kind == CTF_K_INTEGER) || (kind == CTF_K_ENUM)
-	   || (kind == CTF_K_FLOAT))
-	  && ctf_type_reference (fp, id) != CTF_ERR
-	  && ctf_type_encoding (fp, id, enc) == 0);
-}
-
 /* Return a dump for a single type, without member info: but do show the
    type's references.  */
 
@@ -129,26 +115,45 @@ ctf_dump_format_type (ctf_file_t *fp, ctf_id_t id, int flag)
 	  goto err;
 	}
 
-      /* Slices get a different print representation.  */
+      if (asprintf (&bit, " %s%lx: ", nonroot_leader, id) < 0)
+	goto oom;
+      str = str_append (str, bit);
+      free (bit);
+      bit = NULL;
+
+      if (buf[0] != '\0')
+	{
+	  str = str_append (str, buf);
+	  str = str_append (str, " ");
+	}
+
+      free (buf);
+      buf = NULL;
 
-      if (ctf_is_slice (fp, id, &enc))
+      /* Slices get a different print representation.  */
+      if (ctf_type_kind_unsliced (fp, id) == CTF_K_SLICE)
 	{
 	  ctf_type_encoding (fp, id, &enc);
-	  if (asprintf (&bit, " %s%lx: [slice 0x%x:0x%x]%s",
-			nonroot_leader, id, enc.cte_offset, enc.cte_bits,
-			nonroot_trailer) < 0)
+	  if (asprintf (&bit, "[slice 0x%x:0x%x] ",
+			enc.cte_offset, enc.cte_bits) < 0)
 	    goto oom;
 	}
-      else
+      else if (ctf_type_kind (fp, id) == CTF_K_INTEGER)
 	{
-	  if (asprintf (&bit, " %s%lx: %s (size 0x%lx)%s", nonroot_leader,
-			id, buf[0] == '\0' ? "(nameless)" : buf,
-			(unsigned long) ctf_type_size (fp, id),
-			nonroot_trailer) < 0)
+	  ctf_type_encoding (fp, id, &enc);
+	  if (asprintf (&bit, "[0x%x:0x%x] ",
+			enc.cte_offset, enc.cte_bits) < 0)
 	    goto oom;
 	}
-      free (buf);
-      buf = NULL;
+      str = str_append (str, bit);
+      free (bit);
+      bit = NULL;
+
+      if (asprintf (&bit, "(size 0x%lx)%s",
+		    (unsigned long) ctf_type_size (fp, id),
+		    nonroot_trailer) < 0)
+	goto oom;
+
       str = str_append (str, bit);
       free (bit);
       bit = NULL;
@@ -516,6 +521,7 @@ ctf_dump_member (const char *name, ctf_id_t id, unsigned long offset,
   char *typestr = NULL;
   char *bit = NULL;
   ctf_encoding_t ep;
+  int has_encoding = 0;
   ssize_t i;
 
   for (i = 0; i < depth; i++)
@@ -535,24 +541,40 @@ ctf_dump_member (const char *name, ctf_id_t id, unsigned long offset,
 	  return 0;
 	}
 
-      goto oom;
+      return -1;				/* errno is set for us.  */
+    }
+
+  if (ctf_type_encoding (state->cdm_fp, id, &ep) == 0)
+    {
+      has_encoding = 1;
+      ctf_type_encoding (state->cdm_fp, id, &ep);
+
+      if (asprintf (&bit, "    [0x%lx] (ID 0x%lx) (kind %i) %s%s%s:%i "
+		    "(aligned at 0x%lx", offset, id,
+		    ctf_type_kind (state->cdm_fp, id), typestr,
+		    (name[0] != 0 && typestr[0] != 0) ? " " : "", name,
+		    ep.cte_bits, (unsigned long) ctf_type_align (state->cdm_fp,
+								 id)) < 0)
+	goto oom;
+    }
+  else
+    {
+      if (asprintf (&bit, "    [0x%lx] (ID 0x%lx) (kind %i) %s%s%s "
+		    "(aligned at 0x%lx", offset, id,
+		    ctf_type_kind (state->cdm_fp, id), typestr,
+		    (name[0] != 0 && typestr[0] != 0) ? " " : "", name,
+		    (unsigned long) ctf_type_align (state->cdm_fp, id)) < 0)
+	goto oom;
     }
 
-  if (asprintf (&bit, "    [0x%lx] (ID 0x%lx) (kind %i) %s %s (aligned at 0x%lx",
-		offset, id, ctf_type_kind (state->cdm_fp, id), typestr, name,
-		(unsigned long) ctf_type_align (state->cdm_fp, id)) < 0)
-    goto oom;
   *state->cdm_str = str_append (*state->cdm_str, bit);
   free (typestr);
   free (bit);
   typestr = NULL;
   bit = NULL;
 
-  if ((ctf_type_kind (state->cdm_fp, id) == CTF_K_INTEGER)
-      || (ctf_type_kind (state->cdm_fp, id) == CTF_K_FLOAT)
-      || (ctf_is_slice (state->cdm_fp, id, &ep) == CTF_K_ENUM))
+  if (has_encoding)
     {
-      ctf_type_encoding (state->cdm_fp, id, &ep);
       if (asprintf (&bit, ", format 0x%x, offset:bits 0x%x:0x%x", ep.cte_format,
 		    ep.cte_offset, ep.cte_bits) < 0)
 	goto oom;
-- 
2.27.0.247.g3dff7de930

[PATCH 37/59] libctf, open: fix opening CTF in binaries with no symtab

[Nick Alcock]

This is a perfectly possible case, and half of ctf_bfdopen_ctfsect
handled it fine.  The other half hit a divide by zero or two before we
got that far, and had no code path to load the strtab from anywhere
in the absence of a symtab to point at it in any case.

So, as a fallback, if there is no symtab, try loading ".strtab"
explicitly by name, like we used to before we started looking for the
strtab the symtab used.

Of course, such a strtab is not kept hold of by BFD, so this means we
have to bring back the code to possibly explicitly free the strtab that
we read in.

libctf/
	* ctf-impl.h (struct ctf_archive_internal) <ctfi_free_strsect>
	New.
	* ctf-open-bfd.c (ctf_bfdopen_ctfsect): Explicitly open a strtab
	if the input has no symtab, rather than dividing by
	zero. Arrange to free it later via ctfi_free_ctfsect.
	* ctf-archive.c (ctf_new_archive_internal): Do not
	ctfi_free_strsect by default.
	(ctf_arc_close): Possibly free it here.
---
 libctf/ctf-archive.c  |  3 ++
 libctf/ctf-impl.h     |  1 +
 libctf/ctf-open-bfd.c | 81 +++++++++++++++++++++++++++++--------------
 3 files changed, 59 insertions(+), 26 deletions(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 0d824356739..4885e16281e 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -375,6 +375,7 @@ ctf_new_archive_internal (int is_archive, int unmap_on_close,
   if (strsect)
      memcpy (&arci->ctfi_strsect, strsect, sizeof (struct ctf_sect));
   arci->ctfi_free_symsect = 0;
+  arci->ctfi_free_strsect = 0;
   arci->ctfi_unmap_on_close = unmap_on_close;
 
   return arci;
@@ -500,6 +501,8 @@ ctf_arc_close (ctf_archive_t *arc)
     ctf_file_close (arc->ctfi_file);
   if (arc->ctfi_free_symsect)
     free ((void *) arc->ctfi_symsect.cts_data);
+  if (arc->ctfi_free_strsect)
+    free ((void *) arc->ctfi_strsect.cts_data);
   free (arc->ctfi_data);
   if (arc->ctfi_bfd_close)
     arc->ctfi_bfd_close (arc);
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 47a392723e9..913a2647ed2 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -333,6 +333,7 @@ struct ctf_archive_internal
   ctf_sect_t ctfi_symsect;
   ctf_sect_t ctfi_strsect;
   int ctfi_free_symsect;
+  int ctfi_free_strsect;
   void *ctfi_data;
   bfd *ctfi_abfd;		    /* Optional source of section data.  */
   void (*ctfi_bfd_close) (struct ctf_archive_internal *);
diff --git a/libctf/ctf-open-bfd.c b/libctf/ctf-open-bfd.c
index 2d2d572c88f..9fcce2fa9cd 100644
--- a/libctf/ctf-open-bfd.c
+++ b/libctf/ctf-open-bfd.c
@@ -94,46 +94,69 @@ ctf_bfdopen_ctfsect (struct bfd *abfd _libctf_unused_,
   ctf_sect_t *symsectp = NULL;
   ctf_sect_t *strsectp = NULL;
   const char *bfderrstr = NULL;
+  char *strtab_alloc = NULL;
 
 #ifdef HAVE_BFD_ELF
   ctf_sect_t symsect, strsect;
-  Elf_Internal_Shdr *strhdr;
   Elf_Internal_Shdr *symhdr = &elf_symtab_hdr (abfd);
-  size_t symcount = symhdr->sh_size / symhdr->sh_entsize;
+  size_t symcount;
   Elf_Internal_Sym *isymbuf;
-  bfd_byte *symtab;
+  bfd_byte *symtab = NULL;
   const char *strtab = NULL;
+  size_t strsize;
   /* TODO: handle SYMTAB_SHNDX.  */
 
-  if ((symtab = malloc (symhdr->sh_size)) == NULL)
+  /* Get the symtab, and the strtab associated with it.  */
+  if (elf_tdata (abfd) && symhdr && symhdr->sh_size && symhdr->sh_entsize)
     {
-      bfderrstr = "Cannot malloc symbol table";
-      goto err;
-    }
+      symcount = symhdr->sh_size / symhdr->sh_entsize;
+      if ((symtab = malloc (symhdr->sh_size)) == NULL)
+	{
+	  bfderrstr = "Cannot malloc symbol table";
+	  goto err;
+	}
 
-  isymbuf = bfd_elf_get_elf_syms (abfd, symhdr, symcount, 0,
-				  NULL, symtab, NULL);
-  free (isymbuf);
-  if (isymbuf == NULL)
-    {
-      bfderrstr = "Cannot read symbol table";
-      goto err_free_sym;
-    }
+      isymbuf = bfd_elf_get_elf_syms (abfd, symhdr, symcount, 0,
+				      NULL, symtab, NULL);
+      free (isymbuf);
+      if (isymbuf == NULL)
+	{
+	  bfderrstr = "Cannot read symbol table";
+	  goto err_free_sym;
+	}
+
+      if (elf_elfsections (abfd) != NULL
+	  && symhdr->sh_link < elf_numsections (abfd))
+	{
+	  Elf_Internal_Shdr *strhdr = elf_elfsections (abfd)[symhdr->sh_link];
 
-  if (elf_elfsections (abfd) != NULL
-      && symhdr->sh_link < elf_numsections (abfd))
+	  strsize = strhdr->sh_size;
+	  if (strhdr->contents == NULL)
+	    {
+	      if ((strtab = bfd_elf_get_str_section (abfd, symhdr->sh_link)) == NULL)
+		{
+		  bfderrstr = "Cannot read string table";
+		  goto err_free_sym;
+		}
+	    }
+	  else
+	    strtab = (const char *) strhdr->contents;
+	}
+    }
+  else		/* No symtab: just try getting .strtab by name.  */
     {
-      strhdr = elf_elfsections (abfd)[symhdr->sh_link];
-      if (strhdr->contents == NULL)
+      bfd_byte *str_bcontents;
+      asection *str_asect;
+
+      if ((str_asect = bfd_get_section_by_name (abfd, ".strtab")) != NULL)
 	{
-	  if ((strtab = bfd_elf_get_str_section (abfd, symhdr->sh_link)) == NULL)
+	  if (bfd_malloc_and_get_section (abfd, str_asect, &str_bcontents))
 	    {
-	      bfderrstr = "Cannot read string table";
-	      goto err_free_sym;
+	      strtab = (const char *) str_bcontents;
+	      strtab_alloc = (char *) str_bcontents;
+	      strsize = str_asect->size;
 	    }
 	}
-      else
-	strtab = (const char *) strhdr->contents;
     }
 
   if (strtab)
@@ -144,9 +167,12 @@ ctf_bfdopen_ctfsect (struct bfd *abfd _libctf_unused_,
 
       strsect.cts_data = strtab;
       strsect.cts_name = ".strtab";
-      strsect.cts_size = strhdr->sh_size;
+      strsect.cts_size = strsize;
       strsectp = &strsect;
+    }
 
+  if (symtab)
+    {
       assert (symhdr->sh_entsize == get_elf_backend_data (abfd)->s->sizeof_sym);
       symsect.cts_name = ".symtab";
       symsect.cts_entsize = symhdr->sh_entsize;
@@ -159,13 +185,16 @@ ctf_bfdopen_ctfsect (struct bfd *abfd _libctf_unused_,
   arci = ctf_arc_bufopen (ctfsect, symsectp, strsectp, errp);
   if (arci)
     {
-      /* Request freeing of the symsect.  */
+      /* Request freeing of the symsect and possibly the strsect.  */
       arci->ctfi_free_symsect = 1;
+      if (strtab_alloc)
+	arci->ctfi_free_strsect = 1;
       return arci;
     }
 #ifdef HAVE_BFD_ELF
  err_free_sym:
   free (symtab);
+  free (strtab_alloc);
 #endif
 err: _libctf_unused_;
   if (bfderrstr)
-- 
2.27.0.247.g3dff7de930

[PATCH 38/59] libctf, archive: fix bad error message

[Nick Alcock]

Get the function name right.

libctf/
	* ctf-archive.c (ctf_arc_bufopen): Fix message.
---
 libctf/ctf-archive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 4885e16281e..2ec13104d8f 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -411,7 +411,7 @@ ctf_arc_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
       is_archive = 0;
       if ((fp = ctf_bufopen (ctfsect, symsect, strsect, errp)) == NULL)
 	{
-	  ctf_dprintf ("ctf_internal_open(): cannot open CTF: %s\n",
+	  ctf_dprintf ("ctf_arc_bufopen(): cannot open CTF: %s\n",
 		       ctf_errmsg (*errp));
 	  return NULL;
 	}
-- 
2.27.0.247.g3dff7de930

[PATCH 39/59] libctf: check for vasprintf

[Nick Alcock]

We've been using this for all of libctf's history in binutils: we should
check for it in configure.

libctf/
	configure.ac: Check for vasprintf.
	configure: Regenerated.
	config.h.in: Likewise.
---
 libctf/config.h.in  |  4 ++++
 libctf/configure    | 32 +++++++++++++++++++++-----------
 libctf/configure.ac |  2 +-
 3 files changed, 26 insertions(+), 12 deletions(-)

diff --git a/libctf/config.h.in b/libctf/config.h.in
index e663c694d85..897587e5875 100644
--- a/libctf/config.h.in
+++ b/libctf/config.h.in
@@ -29,6 +29,10 @@
    don't. */
 #undef HAVE_DECL_BSWAP_64
 
+/* Define to 1 if you have the declaration of `vasprintf', and to 0 if you
+   don't. */
+#undef HAVE_DECL_VASPRINTF
+
 /* Define to 1 if you have the <dlfcn.h> header file. */
 #undef HAVE_DLFCN_H
 
diff --git a/libctf/configure b/libctf/configure
index 1dc1b65fac3..58aaa3a529b 100755
--- a/libctf/configure
+++ b/libctf/configure
@@ -13099,17 +13099,6 @@ fi
 done
 
 
-ac_fn_c_check_decl "$LINENO" "asprintf" "ac_cv_have_decl_asprintf" "$ac_includes_default"
-if test "x$ac_cv_have_decl_asprintf" = xyes; then :
-  ac_have_decl=1
-else
-  ac_have_decl=0
-fi
-
-cat >>confdefs.h <<_ACEOF
-#define HAVE_DECL_ASPRINTF $ac_have_decl
-_ACEOF
-
 ac_fn_c_check_decl "$LINENO" "bswap_16" "ac_cv_have_decl_bswap_16" "#include <byteswap.h>
 "
 if test "x$ac_cv_have_decl_bswap_16" = xyes; then :
@@ -13144,6 +13133,27 @@ cat >>confdefs.h <<_ACEOF
 #define HAVE_DECL_BSWAP_64 $ac_have_decl
 _ACEOF
 
+ac_fn_c_check_decl "$LINENO" "asprintf" "ac_cv_have_decl_asprintf" "$ac_includes_default"
+if test "x$ac_cv_have_decl_asprintf" = xyes; then :
+  ac_have_decl=1
+else
+  ac_have_decl=0
+fi
+
+cat >>confdefs.h <<_ACEOF
+#define HAVE_DECL_ASPRINTF $ac_have_decl
+_ACEOF
+ac_fn_c_check_decl "$LINENO" "vasprintf" "ac_cv_have_decl_vasprintf" "$ac_includes_default"
+if test "x$ac_cv_have_decl_vasprintf" = xyes; then :
+  ac_have_decl=1
+else
+  ac_have_decl=0
+fi
+
+cat >>confdefs.h <<_ACEOF
+#define HAVE_DECL_VASPRINTF $ac_have_decl
+_ACEOF
+
 
 
 
diff --git a/libctf/configure.ac b/libctf/configure.ac
index f31108c0fad..26b062e7a54 100644
--- a/libctf/configure.ac
+++ b/libctf/configure.ac
@@ -100,9 +100,9 @@ AC_C_BIGENDIAN
 AC_CHECK_HEADERS(byteswap.h endian.h)
 AC_CHECK_FUNCS(pread)
 
-AC_CHECK_DECLS([asprintf])
 dnl Check for bswap_{16,32,64}
 AC_CHECK_DECLS([bswap_16, bswap_32, bswap_64], [], [], [[#include <byteswap.h>]])
+AC_CHECK_DECLS([asprintf, vasprintf])
 
 dnl Check for qsort_r.  (Taken from gnulib.)
 AC_CHECK_FUNCS_ONCE([qsort_r])
-- 
2.27.0.247.g3dff7de930

[PATCH 40/59] libctf: rename the type_mapping_key to type_key

[Nick Alcock]

The name was just annoyingly long and I kept misspelling it.
It's also a bad name: it's not a mapping the type might be *used* in a
type mapping, but it is itself a representation of a type (a ctf_file_t
/ ctf_id_t pair), not of a mapping at all.

libctf/
	* ctf-impl.h (ctf_link_type_mapping_key): Rename to...
	(ctf_link_type_key): ... this, adjusting member prefixes to
	match.
	(ctf_hash_type_mapping_key): Rename to...
	(ctf_hash_type_key): ... this.
	(ctf_hash_eq_type_mapping_key): Rename to...
	(ctf_hash_eq_type_key): ... this.
	* ctf-hash.c (ctf_hash_type_mapping_key): Rename to...
	(ctf_hash_type_key): ... this, and adjust for member name
	changes.
	(ctf_hash_eq_type_mapping_key): Rename to...
	(ctf_hash_eq_type_key): ... this, and adjust for member name
	changes.
	* ctf-link.c (ctf_add_type_mapping): Adjust.  Note the lack of
	need for out-of-memory checking in this code.
	(ctf_type_mapping): Adjust.
---
 libctf/ctf-hash.c | 18 +++++++++---------
 libctf/ctf-impl.h | 19 +++++++++----------
 libctf/ctf-link.c | 21 ++++++++++++---------
 3 files changed, 30 insertions(+), 28 deletions(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index a026ef225da..b9816473a8b 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -94,26 +94,26 @@ ctf_hash_eq_string (const void *a, const void *b)
   return !strcmp((const char *) hep_a->key, (const char *) hep_b->key);
 }
 
-/* Hash a type_mapping_key.  */
+/* Hash a type_key.  */
 unsigned int
-ctf_hash_type_mapping_key (const void *ptr)
+ctf_hash_type_key (const void *ptr)
 {
   ctf_helem_t *hep = (ctf_helem_t *) ptr;
-  ctf_link_type_mapping_key_t *k = (ctf_link_type_mapping_key_t *) hep->key;
+  ctf_link_type_key_t *k = (ctf_link_type_key_t *) hep->key;
 
-  return htab_hash_pointer (k->cltm_fp) + 59 * htab_hash_pointer ((void *) k->cltm_idx);
+  return htab_hash_pointer (k->cltk_fp) + 59 * htab_hash_pointer ((void *) k->cltk_idx);
 }
 
 int
-ctf_hash_eq_type_mapping_key (const void *a, const void *b)
+ctf_hash_eq_type_key (const void *a, const void *b)
 {
   ctf_helem_t *hep_a = (ctf_helem_t *) a;
   ctf_helem_t *hep_b = (ctf_helem_t *) b;
-  ctf_link_type_mapping_key_t *key_a = (ctf_link_type_mapping_key_t *) hep_a->key;
-  ctf_link_type_mapping_key_t *key_b = (ctf_link_type_mapping_key_t *) hep_b->key;
+  ctf_link_type_key_t *key_a = (ctf_link_type_key_t *) hep_a->key;
+  ctf_link_type_key_t *key_b = (ctf_link_type_key_t *) hep_b->key;
 
-  return (key_a->cltm_fp == key_b->cltm_fp)
-    && (key_a->cltm_idx == key_b->cltm_idx);
+  return (key_a->cltk_fp == key_b->cltk_fp)
+    && (key_a->cltk_idx == key_b->cltk_idx);
 }
 
 
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 913a2647ed2..b9d52af9d0e 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -232,16 +232,15 @@ typedef struct ctf_str_atom_ref
   uint32_t *caf_ref;		/* A single ref to this string.  */
 } ctf_str_atom_ref_t;
 
-/* The structure used as the key in a ctf_link_type_mapping, which lets the
-   linker machinery determine which type IDs on the input side of a link map to
-   which types on the output side.  (The value is a ctf_id_t: another
-   index, not a type.)  */
+/* The structure used as the key in a ctf_link_type_mapping.  The value is a
+   type index, not a type ID.  */
 
-typedef struct ctf_link_type_mapping_key
+typedef struct ctf_link_type_key
 {
-  ctf_file_t *cltm_fp;
-  ctf_id_t cltm_idx;
-} ctf_link_type_mapping_key_t;
+  ctf_file_t *cltk_fp;
+  ctf_id_t cltk_idx;
+} ctf_link_type_key_t;
+
 
 /* The ctf_file is the structure used to represent a CTF container to library
    clients, who see it only as an opaque pointer.  Modifications can therefore
@@ -421,12 +420,12 @@ extern ctf_file_t *ctf_get_dict (ctf_file_t *fp, ctf_id_t type);
 typedef unsigned int (*ctf_hash_fun) (const void *ptr);
 extern unsigned int ctf_hash_integer (const void *ptr);
 extern unsigned int ctf_hash_string (const void *ptr);
-extern unsigned int ctf_hash_type_mapping_key (const void *ptr);
+extern unsigned int ctf_hash_type_key (const void *ptr);
 
 typedef int (*ctf_hash_eq_fun) (const void *, const void *);
 extern int ctf_hash_eq_integer (const void *, const void *);
 extern int ctf_hash_eq_string (const void *, const void *);
-extern int ctf_hash_eq_type_mapping_key (const void *, const void *);
+extern int ctf_hash_eq_type_key (const void *, const void *);
 
 extern int ctf_dynset_eq_string (const void *, const void *);
 
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 31179ae13d7..c331fde35dc 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -53,22 +53,25 @@ ctf_add_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type,
 
   if (dst_fp->ctf_link_type_mapping == NULL)
     {
-      ctf_hash_fun f = ctf_hash_type_mapping_key;
-      ctf_hash_eq_fun e = ctf_hash_eq_type_mapping_key;
+      ctf_hash_fun f = ctf_hash_type_key;
+      ctf_hash_eq_fun e = ctf_hash_eq_type_key;
 
       if ((dst_fp->ctf_link_type_mapping = ctf_dynhash_create (f, e, free,
 							       NULL)) == NULL)
 	return;
     }
 
-  ctf_link_type_mapping_key_t *key;
-  key = calloc (1, sizeof (struct ctf_link_type_mapping_key));
+  ctf_link_type_key_t *key;
+  key = calloc (1, sizeof (struct ctf_link_type_key));
   if (!key)
     return;
 
-  key->cltm_fp = src_fp;
-  key->cltm_idx = src_type;
+  key->cltk_fp = src_fp;
+  key->cltk_idx = src_type;
 
+  /* No OOM checking needed, because if this doesn't work the worst we'll do is
+     add a few more duplicate types (which will probably run out of memory
+     anyway).  */
   ctf_dynhash_insert (dst_fp->ctf_link_type_mapping, key,
 		      (void *) (uintptr_t) dst_type);
 }
@@ -78,7 +81,7 @@ ctf_add_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type,
 ctf_id_t
 ctf_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type, ctf_file_t **dst_fp)
 {
-  ctf_link_type_mapping_key_t key;
+  ctf_link_type_key_t key;
   ctf_file_t *target_fp = *dst_fp;
   ctf_id_t dst_type = 0;
 
@@ -86,8 +89,8 @@ ctf_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type, ctf_file_t **dst_fp)
     src_fp = src_fp->ctf_parent;
 
   src_type = LCTF_TYPE_TO_INDEX(src_fp, src_type);
-  key.cltm_fp = src_fp;
-  key.cltm_idx = src_type;
+  key.cltk_fp = src_fp;
+  key.cltk_idx = src_type;
 
   if (target_fp->ctf_link_type_mapping)
     dst_type = (uintptr_t) ctf_dynhash_lookup (target_fp->ctf_link_type_mapping,
-- 
2.27.0.247.g3dff7de930

[PATCH 41/59] libctf: sort out potential refcount loops

[Nick Alcock]

When you link TUs that contain conflicting types together, the resulting
CTF section is an archive containing many CTF dicts.  These dicts appear
in ctf_link_outputs of the shared dict, with each ctf_import'ing that
shared dict.  ctf_importing a dict bumps its refcount to stop it going
away while it's in use -- but if the shared dict (whose refcount is
bumped) has the child dict (doing the bumping) in its ctf_link_outputs,
we have a refcount loop, since the child dict only un-ctf_imports and
drops the parent's refcount when it is freed, but the child is only
freed when the parent's refcount falls to zero.

(In the future, this will be able to go wrong on the inputs too, when an
ld -r'ed deduplicated output with conflicts is relinked.  Right now this
cannot happen because we don't ctf_import such dicts at all.  This will
be fixed in a later commit in this series.)

Fix this by introducing an internal-use-only ctf_import_unref function
that imports a parent dict *witthout* bumping the parent's refcount, and
using it when we create per-CU outputs.  This function is only safe to
use if you know the parent cannot go away while the child exists: but if
the parent *owns* the child, as here, this is necessarily true.

Record in the ctf_file_t whether a parent was imported via ctf_import or
ctf_import_unref, so that if you do another ctf_import later on (or a
ctf_import_unref) it can decide whether to drop the refcount of the
existing parent being replaced depending on which function you used to
import that one.  Adjust ctf_serialize so that rather than doing a
ctf_import (which is wrong if the original import was
ctf_import_unref'fed), we just copy the parent field and refcount over
and forcibly flip the unref flag on on the old copy we are going to
discard.

ctf_file_close also needs a bit of tweaking to only close the parent if
it was not imported with ctf_import_unref: while we're at it, guard
against repeated closes with a refcount of zero and stop them causing
double-frees, even if destruction of things freed *inside*
ctf_file_close cause such recursion.

Verified no leaks or accesses to freed memory after all of this with
valgrind.  (It was leak-happy before.)

libctf/
	* ctf-impl.c (ctf_file_t) <ctf_parent_unreffed>: New.
	(ctf_import_unref): New.
	* ctf-open.c (ctf_file_close) Drop the refcount all the way to
	zero.  Don't recurse back in if the refcount is already zero.
	(ctf_import): Check ctf_parent_unreffed before deciding whether
	to close a pre-existing parent.  Set it to zero.
	(ctf_import_unreffed): New, as above, setting
	ctf_parent_unreffed to 1.
	* ctf-create.c (ctf_serialize): Do not ctf_import into the new
	child: use direct assignment, and set unreffed on the new and
	old children.
	* ctf-link.c (ctf_create_per_cu): Import the parent using
	ctf_import_unreffed.
---
 libctf/ctf-create.c |  4 +++-
 libctf/ctf-impl.h   |  2 ++
 libctf/ctf-link.c   |  2 +-
 libctf/ctf-open.c   | 51 +++++++++++++++++++++++++++++++++++++++------
 4 files changed, 51 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index a538b2d5603..a964c20b9ed 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -516,8 +516,9 @@ ctf_serialize (ctf_file_t *fp)
     }
 
   (void) ctf_setmodel (nfp, ctf_getmodel (fp));
-  (void) ctf_import (nfp, fp->ctf_parent);
 
+  nfp->ctf_parent = fp->ctf_parent;
+  nfp->ctf_parent_unreffed = fp->ctf_parent_unreffed;
   nfp->ctf_refcnt = fp->ctf_refcnt;
   nfp->ctf_flags |= fp->ctf_flags & ~LCTF_DIRTY;
   if (nfp->ctf_dynbase == NULL)
@@ -565,6 +566,7 @@ ctf_serialize (ctf_file_t *fp)
   fp->ctf_syn_ext_strtab = NULL;
   fp->ctf_link_cu_mapping = NULL;
   fp->ctf_link_type_mapping = NULL;
+  fp->ctf_parent_unreffed = 1;
 
   fp->ctf_dvhash = NULL;
   memset (&fp->ctf_dvdefs, 0, sizeof (ctf_list_t));
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index b9d52af9d0e..4c8a37c4c26 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -291,6 +291,7 @@ struct ctf_file
   const char *ctf_cuname;	  /* Compilation unit name (if any).  */
   char *ctf_dyncuname;		  /* Dynamically allocated name of CU.  */
   struct ctf_file *ctf_parent;	  /* Parent CTF container (if any).  */
+  int ctf_parent_unreffed;	  /* Parent set by ctf_import_unref?  */
   const char *ctf_parlabel;	  /* Label in parent container (if any).  */
   const char *ctf_parname;	  /* Basename of parent (if any).  */
   char *ctf_dynparname;		  /* Dynamically allocated name of parent.  */
@@ -536,6 +537,7 @@ extern ctf_file_t *ctf_simple_open_internal (const char *, size_t, const char *,
 extern ctf_file_t *ctf_bufopen_internal (const ctf_sect_t *, const ctf_sect_t *,
 					 const ctf_sect_t *, ctf_dynhash_t *,
 					 int, int *);
+extern int ctf_import_unref (ctf_file_t *fp, ctf_file_t *pfp);
 extern int ctf_serialize (ctf_file_t *);
 
 _libctf_malloc_
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index c331fde35dc..bcfd2564fbc 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -222,7 +222,7 @@ ctf_create_per_cu (ctf_file_t *fp, const char *filename, const char *cuname)
       if (ctf_dynhash_insert (fp->ctf_link_outputs, dynname, cu_fp) < 0)
 	goto oom;
 
-      ctf_import (cu_fp, fp);
+      ctf_import_unref (cu_fp, fp);
       ctf_cuname_set (cu_fp, cuname);
       ctf_parent_name_set (cu_fp, _CTF_SECTION);
     }
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index 24899f08e20..87bff2f122d 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1657,9 +1657,17 @@ ctf_file_close (ctf_file_t *fp)
       return;
     }
 
+  /* It is possible to recurse back in here, notably if dicts in the
+     ctf_link_inputs or ctf_link_outputs cite this dict as a parent without
+     using ctf_import_unref.  Do nothing in that case.  */
+  if (fp->ctf_refcnt == 0)
+    return;
+
+  fp->ctf_refcnt--;
   free (fp->ctf_dyncuname);
   free (fp->ctf_dynparname);
-  ctf_file_close (fp->ctf_parent);
+  if (fp->ctf_parent && !fp->ctf_parent_unreffed)
+    ctf_file_close (fp->ctf_parent);
 
   for (dtd = ctf_list_next (&fp->ctf_dtdefs); dtd != NULL; dtd = ntd)
     {
@@ -1816,13 +1824,44 @@ ctf_import (ctf_file_t *fp, ctf_file_t *pfp)
   if (pfp != NULL && pfp->ctf_dmodel != fp->ctf_dmodel)
     return (ctf_set_errno (fp, ECTF_DMODEL));
 
-  if (fp->ctf_parent != NULL)
+  if (fp->ctf_parent && !fp->ctf_parent_unreffed)
+    ctf_file_close (fp->ctf_parent);
+  fp->ctf_parent = NULL;
+
+  if (pfp != NULL)
     {
-      fp->ctf_parent->ctf_refcnt--;
-      ctf_file_close (fp->ctf_parent);
-      fp->ctf_parent = NULL;
+      int err;
+
+      if (fp->ctf_parname == NULL)
+	if ((err = ctf_parent_name_set (fp, "PARENT")) < 0)
+	  return err;
+
+      fp->ctf_flags |= LCTF_CHILD;
+      pfp->ctf_refcnt++;
+      fp->ctf_parent_unreffed = 0;
     }
 
+  fp->ctf_parent = pfp;
+  return 0;
+}
+
+/* Like ctf_import, but does not increment the refcount on the imported parent
+   or close it at any point: as a result it can go away at any time and the
+   caller must do all freeing itself.  Used internally to avoid refcount
+   loops.  */
+int
+ctf_import_unref (ctf_file_t *fp, ctf_file_t *pfp)
+{
+  if (fp == NULL || fp == pfp || (pfp != NULL && pfp->ctf_refcnt == 0))
+    return (ctf_set_errno (fp, EINVAL));
+
+  if (pfp != NULL && pfp->ctf_dmodel != fp->ctf_dmodel)
+    return (ctf_set_errno (fp, ECTF_DMODEL));
+
+  if (fp->ctf_parent && !fp->ctf_parent_unreffed)
+    ctf_file_close (fp->ctf_parent);
+  fp->ctf_parent = NULL;
+
   if (pfp != NULL)
     {
       int err;
@@ -1832,7 +1871,7 @@ ctf_import (ctf_file_t *fp, ctf_file_t *pfp)
 	  return err;
 
       fp->ctf_flags |= LCTF_CHILD;
-      pfp->ctf_refcnt++;
+      fp->ctf_parent_unreffed = 1;
     }
 
   fp->ctf_parent = pfp;
-- 
2.27.0.247.g3dff7de930

[PATCH 42/59] libctf: drop error-prone ctf_strerror

[Nick Alcock]

This utility function is almost useless (all it does is casts the result
of a strerror) but has a seriously confusing name.  Over and over again
I have accidentally called it instead of ctf_errmsg, and hidden a
time-bomb for myself in a hard-to-test error-handling path: since
ctf_strerror is just a strerror wrapper, it cannot handle CTF errnos,
unlike ctf_errmsg.  It's astonishingly lucky that none of these errors
have crept into any commits to date.

Fuse it into ctf_errmsg and drop it.

libctf/
	* ctf-impl.h (ctf_strerror): Delete.
	* ctf-subr.c (ctf_strerror): Likewise.
	* ctf-error.c (ctf_errmsg): Stop using ctf_strerror: just use
	strerror directly.
---
 libctf/ctf-error.c | 3 ++-
 libctf/ctf-impl.h  | 1 -
 libctf/ctf-subr.c  | 6 ------
 3 files changed, 2 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-error.c b/libctf/ctf-error.c
index 1e69672007a..ac42784df65 100644
--- a/libctf/ctf-error.c
+++ b/libctf/ctf-error.c
@@ -19,6 +19,7 @@
 
 #include <ctf-impl.h>
 #include <stddef.h>
+#include <string.h>
 
 /* This construct is due to Bruno Haible: much thanks.  */
 
@@ -67,7 +68,7 @@ ctf_errmsg (int error)
   if (error >= ECTF_BASE && (error - ECTF_BASE) < ECTF_NERR)
     str = _ctf_errlist.str + _ctf_erridx[error - ECTF_BASE];
   else
-    str = ctf_strerror (error);
+    str = (const char *) strerror (error);
 
   return (str ? str : "Unknown error");
 }
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 4c8a37c4c26..71b732a2775 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -548,7 +548,6 @@ extern ssize_t ctf_pread (int fd, void *buf, ssize_t count, off_t offset);
 extern void *ctf_realloc (ctf_file_t *, void *, size_t);
 extern char *ctf_str_append (char *, const char *);
 extern char *ctf_str_append_noerr (char *, const char *);
-extern const char *ctf_strerror (int);
 
 extern ctf_id_t ctf_type_resolve_unsliced (ctf_file_t *, ctf_id_t);
 extern int ctf_type_kind_unsliced (ctf_file_t *, ctf_id_t);
diff --git a/libctf/ctf-subr.c b/libctf/ctf-subr.c
index 0b49ae9fca8..455f18dcffb 100644
--- a/libctf/ctf-subr.c
+++ b/libctf/ctf-subr.c
@@ -122,12 +122,6 @@ ctf_pread (int fd, void *buf, ssize_t count, off_t offset)
   return acc;
 }
 
-const char *
-ctf_strerror (int err)
-{
-  return (const char *) (strerror (err));
-}
-
 /* Set the CTF library client version to the specified version.  If version is
    zero, we just return the default library version number.  */
 int
-- 
2.27.0.247.g3dff7de930

[PATCH 43/59] libctf, link: add lazy linking: clean up input members: err/warn cleanup

[Nick Alcock]

This rather large and intertwined pile of changes does three things:

First, it transitions from dprintf to ctf_err_warn for things the user might
care about: this one file is the major impetus for the ctf_err_warn
infrastructure, because things like file names are crucial in linker
error messages, and errno values are utterly incapable of
communicating them

Second, it stabilizes the ctf_link APIs: you can now call
ctf_link_add_ctf without a CTF argument (only a NAME), to lazily
ctf_open the file with the given NAME when needed, and close it as soon
as possible, to save memory.  This is not an API change because a null
CTF argument was prohibited before now.

Since getting CTF directly from files uses ctf_open, passing in only a
NAME requires use of libctf, not libctf-nobfd.  The linker's behaviour
is unchanged, as it still passes in a ctf_archive_t as before.

This also let us fix a leak: we were opening ctf_archives and their
containing ctf_files, then only closing the files and leaving the
archives open.

Third, this commit restructures the ctf_link_in_member argument used by
the CTF linking machinery and adjusts its users accordingly.

We drop two members:

- arcname, which is difficult to construct and then only used in error
  messages (that were only dprintf()ed, so never seen!)
- share_mode, since we store the flags passed to ctf_link (including the
  share mode) in a new ctf_file_t.ctf_link_flags to help dedup get hold
  of it

We rename others whose existing names were fairly dreadful:

- done_main_member -> done_parent, using consistent terminology for .ctf
  as the parent of all archive members
- main_input_fp -> in_fp_parent, likewise
- file_name -> in_file_name, likewise

We add one new member, cu_mapped.

Finally, we move the various frees of things like mapping table data to
the top-level ctf_link, since deduplicating links will want to do that
too.

include/
	* ctf-api.h (ECTF_NEEDSBFD): New.
	(ECTF_NERR): Adjust.
	(ctf_link): Rename share_mode arg to flags.
libctf/
	* Makefile.am: Set -DNOBFD=1 in libctf-nobfd, and =0 elsewhere.
	* Makefile.in: Regenerated.
	* ctf-impl.h (ctf_link_input_name): New.
	(ctf_file_t) <ctf_link_flags>: New.
	* ctf-create.c (ctf_serialize): Adjust accordingly.
	* ctf-link.c: Define ctf_open as weak when PIC.
	(ctf_arc_close_thunk): Remove unnecessary thunk.
	(ctf_file_close_thunk): Likewise.
	(ctf_link_input_name): New.
	(ctf_link_input_t): New value of the ctf_file_t.ctf_link_input.
	(ctf_link_input_close): Adjust accordingly.
	(ctf_link_add_ctf_internal): New, split from...
	(ctf_link_add_ctf): ... here.  Return error if lazy loading of
	CTF is not possible.  Change to just call...
	(ctf_link_add): ... this new function.
	(ctf_link_add_cu_mapping): Transition to ctf_err_warn.  Drop the
	ctf_file_close_thunk.
	(ctf_link_in_member_cb_arg_t) <file_name> Rename to...
	<in_file_name>: ... this.
	<arcname>: Drop.
	<share_mode>: Likewise (migrated to ctf_link_flags).
	<done_main_member>: Rename to...
	<done_parent>: ... this.
	<main_input_fp>: Rename to...
	<in_fp_parent>: ... this.
	<cu_mapped>: New.
	(ctf_link_one_type): Adjuwt accordingly.  Transition to
	ctf_err_warn, removing a TODO.
	(ctf_link_one_variable): Note a case too common to warn about.
	Report in the debug stream if a cu-mapped link prevents addition
	of a conflicting variable.
	(ctf_link_one_input_archive_member): Adjust.
	(ctf_link_lazy_open): New, open a CTF archive for linking when
	needed.
	(ctf_link_close_one_input_archive): New, close it again.
	(ctf_link_one_input_archive): Adjust for lazy opening, member
	renames, and ctf_err_warn transition.  Move the
	empty_link_type_mapping call to...
	(ctf_link): ... here.  Adjut for renamings and thunk removal.
	Don't spuriously fail if some input contains no CTF data.
	(ctf_link_write): ctf_err_warn transition.
	* libctf.ver: Remove not-yet-stable comment.
---
 include/ctf-api.h   |   9 +-
 libctf/Makefile.am  |   2 +
 libctf/Makefile.in  | 294 +++++++++++++++++++++++++++++++---
 libctf/ctf-create.c |   1 +
 libctf/ctf-impl.h   |   4 +
 libctf/ctf-link.c   | 381 ++++++++++++++++++++++++++++++++------------
 libctf/libctf.ver   |   2 -
 7 files changed, 560 insertions(+), 133 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index e061b7022b6..7d3e1c8bfcd 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -208,10 +208,11 @@ enum
    ECTF_NEXT_END,	/* End of iteration.  */
    ECTF_NEXT_WRONGFUN,	/* Wrong iteration function called.  */
    ECTF_NEXT_WRONGFP,	/* Iteration entity changed in mid-iterate.  */
-   ECTF_FLAGS		/* CTF header contains flags unknown to libctf.  */
+   ECTF_FLAGS,		/* CTF header contains flags unknown to libctf.  */
+   ECTF_NEEDSBFD	/* This feature needs a libctf with BFD support.  */
   };
 
-#define ECTF_NERR (ECTF_FLAGS - ECTF_BASE + 1)	/* Count of CTF errors.  */
+#define ECTF_NERR (ECTF_NEEDSBFD - ECTF_BASE + 1) /* Count of CTF errors.  */
 
 /* The CTF data model is inferred to be the caller's data model or the data
    model of the given object, unless ctf_setmodel() is explicitly called.  */
@@ -452,10 +453,8 @@ extern int ctf_gzwrite (ctf_file_t *fp, gzFile fd);
 extern int ctf_compress_write (ctf_file_t * fp, int fd);
 extern unsigned char *ctf_write_mem (ctf_file_t *, size_t *, size_t threshold);
 
-/* The ctf_link interfaces are not stable yet.  No guarantees!  */
-
 extern int ctf_link_add_ctf (ctf_file_t *, ctf_archive_t *, const char *);
-extern int ctf_link (ctf_file_t *, int share_mode);
+extern int ctf_link (ctf_file_t *, int flags);
 typedef const char *ctf_link_strtab_string_f (uint32_t *offset, void *arg);
 extern int ctf_link_add_strtab (ctf_file_t *, ctf_link_strtab_string_f *,
 				void *);
diff --git a/libctf/Makefile.am b/libctf/Makefile.am
index cdd3a5c55d6..de27cf82394 100644
--- a/libctf/Makefile.am
+++ b/libctf/Makefile.am
@@ -41,6 +41,7 @@ endif
 
 libctf_nobfd_la_LIBADD = @SHARED_LIBADD@ $(ZLIB)
 libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
+libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c ctf-decl.c ctf-error.c \
 			  ctf-hash.c ctf-labels.c ctf-link.c ctf-lookup.c ctf-open.c \
 			  ctf-string.c ctf-subr.c ctf-types.c ctf-util.c
@@ -49,6 +50,7 @@ libctf_nobfd_la_SOURCES += ctf-qsort_r.c
 endif
 
 libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
+libctf_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=0
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
 libctf_la_LDFLAGS = $(libctf_nobfd_la_LDFLAGS)
 libctf_la_SOURCES = $(libctf_nobfd_la_SOURCES) ctf-open-bfd.c
diff --git a/libctf/Makefile.in b/libctf/Makefile.in
index a26fa31da49..3b7feea02d2 100644
--- a/libctf/Makefile.in
+++ b/libctf/Makefile.in
@@ -167,11 +167,15 @@ am__libctf_nobfd_la_SOURCES_DIST = ctf-archive.c ctf-dump.c \
 	ctf-create.c ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c \
 	ctf-link.c ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c \
 	ctf-types.c ctf-util.c ctf-qsort_r.c
-@NEED_CTF_QSORT_R_TRUE@am__objects_1 = ctf-qsort_r.lo
-am_libctf_nobfd_la_OBJECTS = ctf-archive.lo ctf-dump.lo ctf-create.lo \
-	ctf-decl.lo ctf-error.lo ctf-hash.lo ctf-labels.lo ctf-link.lo \
-	ctf-lookup.lo ctf-open.lo ctf-string.lo ctf-subr.lo \
-	ctf-types.lo ctf-util.lo $(am__objects_1)
+@NEED_CTF_QSORT_R_TRUE@am__objects_1 = libctf_nobfd_la-ctf-qsort_r.lo
+am_libctf_nobfd_la_OBJECTS = libctf_nobfd_la-ctf-archive.lo \
+	libctf_nobfd_la-ctf-dump.lo libctf_nobfd_la-ctf-create.lo \
+	libctf_nobfd_la-ctf-decl.lo libctf_nobfd_la-ctf-error.lo \
+	libctf_nobfd_la-ctf-hash.lo libctf_nobfd_la-ctf-labels.lo \
+	libctf_nobfd_la-ctf-link.lo libctf_nobfd_la-ctf-lookup.lo \
+	libctf_nobfd_la-ctf-open.lo libctf_nobfd_la-ctf-string.lo \
+	libctf_nobfd_la-ctf-subr.lo libctf_nobfd_la-ctf-types.lo \
+	libctf_nobfd_la-ctf-util.lo $(am__objects_1)
 libctf_nobfd_la_OBJECTS = $(am_libctf_nobfd_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
 am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@)
@@ -188,11 +192,15 @@ am__libctf_la_SOURCES_DIST = ctf-archive.c ctf-dump.c ctf-create.c \
 	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
 	ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c ctf-types.c \
 	ctf-util.c ctf-qsort_r.c ctf-open-bfd.c
-am__objects_2 = ctf-archive.lo ctf-dump.lo ctf-create.lo ctf-decl.lo \
-	ctf-error.lo ctf-hash.lo ctf-labels.lo ctf-link.lo \
-	ctf-lookup.lo ctf-open.lo ctf-string.lo ctf-subr.lo \
-	ctf-types.lo ctf-util.lo $(am__objects_1)
-am_libctf_la_OBJECTS = $(am__objects_2) ctf-open-bfd.lo
+@NEED_CTF_QSORT_R_TRUE@am__objects_2 = libctf_la-ctf-qsort_r.lo
+am__objects_3 = libctf_la-ctf-archive.lo libctf_la-ctf-dump.lo \
+	libctf_la-ctf-create.lo libctf_la-ctf-decl.lo \
+	libctf_la-ctf-error.lo libctf_la-ctf-hash.lo \
+	libctf_la-ctf-labels.lo libctf_la-ctf-link.lo \
+	libctf_la-ctf-lookup.lo libctf_la-ctf-open.lo \
+	libctf_la-ctf-string.lo libctf_la-ctf-subr.lo \
+	libctf_la-ctf-types.lo libctf_la-ctf-util.lo $(am__objects_2)
+am_libctf_la_OBJECTS = $(am__objects_3) libctf_la-ctf-open-bfd.lo
 libctf_la_OBJECTS = $(am_libctf_la_OBJECTS)
 libctf_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
 	$(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \
@@ -444,11 +452,13 @@ AM_CFLAGS = -std=gnu99 @ac_libctf_warn_cflags@ @warn@ @c_warn@ @WARN_PEDANTIC@ @
 @INSTALL_LIBBFD_FALSE@noinst_LTLIBRARIES = libctf.la libctf-nobfd.la
 libctf_nobfd_la_LIBADD = @SHARED_LIBADD@ $(ZLIB)
 libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
+libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c \
 	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
 	ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c ctf-types.c \
 	ctf-util.c $(am__append_1)
 libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
+libctf_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=0
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
 libctf_la_LDFLAGS = $(libctf_nobfd_la_LDFLAGS)
 libctf_la_SOURCES = $(libctf_nobfd_la_SOURCES) ctf-open-bfd.c
@@ -565,22 +575,37 @@ mostlyclean-compile:
 distclean-compile:
 	-rm -f *.tab.c
 
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-archive.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-create.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-decl.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-dump.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-error.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-hash.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-labels.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-link.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-lookup.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-open-bfd.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-open.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-qsort_r.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-string.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-subr.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-types.Plo@am__quote@
-@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ctf-util.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-archive.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-create.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-decl.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-dump.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-error.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-hash.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-labels.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-link.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-lookup.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-open-bfd.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-open.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-qsort_r.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-string.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-subr.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-types.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-util.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-archive.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-create.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-decl.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-dump.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-error.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-hash.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-labels.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-link.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-lookup.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-open.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-qsort_r.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-string.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-subr.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-types.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-util.Plo@am__quote@
 
 .c.o:
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $<
@@ -603,6 +628,223 @@ distclean-compile:
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $<
 
+libctf_nobfd_la-ctf-archive.lo: ctf-archive.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-archive.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-archive.Tpo -c -o libctf_nobfd_la-ctf-archive.lo `test -f 'ctf-archive.c' || echo '$(srcdir)/'`ctf-archive.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-archive.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-archive.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-archive.c' object='libctf_nobfd_la-ctf-archive.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-archive.lo `test -f 'ctf-archive.c' || echo '$(srcdir)/'`ctf-archive.c
+
+libctf_nobfd_la-ctf-dump.lo: ctf-dump.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-dump.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-dump.Tpo -c -o libctf_nobfd_la-ctf-dump.lo `test -f 'ctf-dump.c' || echo '$(srcdir)/'`ctf-dump.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-dump.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-dump.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-dump.c' object='libctf_nobfd_la-ctf-dump.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-dump.lo `test -f 'ctf-dump.c' || echo '$(srcdir)/'`ctf-dump.c
+
+libctf_nobfd_la-ctf-create.lo: ctf-create.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-create.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-create.Tpo -c -o libctf_nobfd_la-ctf-create.lo `test -f 'ctf-create.c' || echo '$(srcdir)/'`ctf-create.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-create.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-create.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-create.c' object='libctf_nobfd_la-ctf-create.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-create.lo `test -f 'ctf-create.c' || echo '$(srcdir)/'`ctf-create.c
+
+libctf_nobfd_la-ctf-decl.lo: ctf-decl.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-decl.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-decl.Tpo -c -o libctf_nobfd_la-ctf-decl.lo `test -f 'ctf-decl.c' || echo '$(srcdir)/'`ctf-decl.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-decl.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-decl.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-decl.c' object='libctf_nobfd_la-ctf-decl.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-decl.lo `test -f 'ctf-decl.c' || echo '$(srcdir)/'`ctf-decl.c
+
+libctf_nobfd_la-ctf-error.lo: ctf-error.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-error.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-error.Tpo -c -o libctf_nobfd_la-ctf-error.lo `test -f 'ctf-error.c' || echo '$(srcdir)/'`ctf-error.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-error.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-error.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-error.c' object='libctf_nobfd_la-ctf-error.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-error.lo `test -f 'ctf-error.c' || echo '$(srcdir)/'`ctf-error.c
+
+libctf_nobfd_la-ctf-hash.lo: ctf-hash.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-hash.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-hash.Tpo -c -o libctf_nobfd_la-ctf-hash.lo `test -f 'ctf-hash.c' || echo '$(srcdir)/'`ctf-hash.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-hash.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-hash.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-hash.c' object='libctf_nobfd_la-ctf-hash.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-hash.lo `test -f 'ctf-hash.c' || echo '$(srcdir)/'`ctf-hash.c
+
+libctf_nobfd_la-ctf-labels.lo: ctf-labels.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-labels.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-labels.Tpo -c -o libctf_nobfd_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-labels.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-labels.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-labels.c' object='libctf_nobfd_la-ctf-labels.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
+
+libctf_nobfd_la-ctf-link.lo: ctf-link.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-link.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-link.Tpo -c -o libctf_nobfd_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-link.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-link.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-link.c' object='libctf_nobfd_la-ctf-link.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
+
+libctf_nobfd_la-ctf-lookup.lo: ctf-lookup.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-lookup.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-lookup.Tpo -c -o libctf_nobfd_la-ctf-lookup.lo `test -f 'ctf-lookup.c' || echo '$(srcdir)/'`ctf-lookup.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-lookup.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-lookup.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-lookup.c' object='libctf_nobfd_la-ctf-lookup.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-lookup.lo `test -f 'ctf-lookup.c' || echo '$(srcdir)/'`ctf-lookup.c
+
+libctf_nobfd_la-ctf-open.lo: ctf-open.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-open.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-open.Tpo -c -o libctf_nobfd_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-open.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-open.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-open.c' object='libctf_nobfd_la-ctf-open.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
+
+libctf_nobfd_la-ctf-string.lo: ctf-string.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-string.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-string.Tpo -c -o libctf_nobfd_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-string.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-string.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-string.c' object='libctf_nobfd_la-ctf-string.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
+
+libctf_nobfd_la-ctf-subr.lo: ctf-subr.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-subr.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-subr.Tpo -c -o libctf_nobfd_la-ctf-subr.lo `test -f 'ctf-subr.c' || echo '$(srcdir)/'`ctf-subr.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-subr.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-subr.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-subr.c' object='libctf_nobfd_la-ctf-subr.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-subr.lo `test -f 'ctf-subr.c' || echo '$(srcdir)/'`ctf-subr.c
+
+libctf_nobfd_la-ctf-types.lo: ctf-types.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-types.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-types.Tpo -c -o libctf_nobfd_la-ctf-types.lo `test -f 'ctf-types.c' || echo '$(srcdir)/'`ctf-types.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-types.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-types.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-types.c' object='libctf_nobfd_la-ctf-types.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-types.lo `test -f 'ctf-types.c' || echo '$(srcdir)/'`ctf-types.c
+
+libctf_nobfd_la-ctf-util.lo: ctf-util.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-util.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-util.Tpo -c -o libctf_nobfd_la-ctf-util.lo `test -f 'ctf-util.c' || echo '$(srcdir)/'`ctf-util.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-util.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-util.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-util.c' object='libctf_nobfd_la-ctf-util.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-util.lo `test -f 'ctf-util.c' || echo '$(srcdir)/'`ctf-util.c
+
+libctf_nobfd_la-ctf-qsort_r.lo: ctf-qsort_r.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-qsort_r.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-qsort_r.Tpo -c -o libctf_nobfd_la-ctf-qsort_r.lo `test -f 'ctf-qsort_r.c' || echo '$(srcdir)/'`ctf-qsort_r.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-qsort_r.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-qsort_r.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-qsort_r.c' object='libctf_nobfd_la-ctf-qsort_r.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-qsort_r.lo `test -f 'ctf-qsort_r.c' || echo '$(srcdir)/'`ctf-qsort_r.c
+
+libctf_la-ctf-archive.lo: ctf-archive.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-archive.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-archive.Tpo -c -o libctf_la-ctf-archive.lo `test -f 'ctf-archive.c' || echo '$(srcdir)/'`ctf-archive.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-archive.Tpo $(DEPDIR)/libctf_la-ctf-archive.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-archive.c' object='libctf_la-ctf-archive.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-archive.lo `test -f 'ctf-archive.c' || echo '$(srcdir)/'`ctf-archive.c
+
+libctf_la-ctf-dump.lo: ctf-dump.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-dump.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-dump.Tpo -c -o libctf_la-ctf-dump.lo `test -f 'ctf-dump.c' || echo '$(srcdir)/'`ctf-dump.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-dump.Tpo $(DEPDIR)/libctf_la-ctf-dump.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-dump.c' object='libctf_la-ctf-dump.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-dump.lo `test -f 'ctf-dump.c' || echo '$(srcdir)/'`ctf-dump.c
+
+libctf_la-ctf-create.lo: ctf-create.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-create.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-create.Tpo -c -o libctf_la-ctf-create.lo `test -f 'ctf-create.c' || echo '$(srcdir)/'`ctf-create.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-create.Tpo $(DEPDIR)/libctf_la-ctf-create.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-create.c' object='libctf_la-ctf-create.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-create.lo `test -f 'ctf-create.c' || echo '$(srcdir)/'`ctf-create.c
+
+libctf_la-ctf-decl.lo: ctf-decl.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-decl.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-decl.Tpo -c -o libctf_la-ctf-decl.lo `test -f 'ctf-decl.c' || echo '$(srcdir)/'`ctf-decl.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-decl.Tpo $(DEPDIR)/libctf_la-ctf-decl.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-decl.c' object='libctf_la-ctf-decl.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-decl.lo `test -f 'ctf-decl.c' || echo '$(srcdir)/'`ctf-decl.c
+
+libctf_la-ctf-error.lo: ctf-error.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-error.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-error.Tpo -c -o libctf_la-ctf-error.lo `test -f 'ctf-error.c' || echo '$(srcdir)/'`ctf-error.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-error.Tpo $(DEPDIR)/libctf_la-ctf-error.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-error.c' object='libctf_la-ctf-error.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-error.lo `test -f 'ctf-error.c' || echo '$(srcdir)/'`ctf-error.c
+
+libctf_la-ctf-hash.lo: ctf-hash.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-hash.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-hash.Tpo -c -o libctf_la-ctf-hash.lo `test -f 'ctf-hash.c' || echo '$(srcdir)/'`ctf-hash.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-hash.Tpo $(DEPDIR)/libctf_la-ctf-hash.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-hash.c' object='libctf_la-ctf-hash.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-hash.lo `test -f 'ctf-hash.c' || echo '$(srcdir)/'`ctf-hash.c
+
+libctf_la-ctf-labels.lo: ctf-labels.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-labels.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-labels.Tpo -c -o libctf_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-labels.Tpo $(DEPDIR)/libctf_la-ctf-labels.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-labels.c' object='libctf_la-ctf-labels.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
+
+libctf_la-ctf-link.lo: ctf-link.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-link.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-link.Tpo -c -o libctf_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-link.Tpo $(DEPDIR)/libctf_la-ctf-link.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-link.c' object='libctf_la-ctf-link.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
+
+libctf_la-ctf-lookup.lo: ctf-lookup.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-lookup.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-lookup.Tpo -c -o libctf_la-ctf-lookup.lo `test -f 'ctf-lookup.c' || echo '$(srcdir)/'`ctf-lookup.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-lookup.Tpo $(DEPDIR)/libctf_la-ctf-lookup.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-lookup.c' object='libctf_la-ctf-lookup.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-lookup.lo `test -f 'ctf-lookup.c' || echo '$(srcdir)/'`ctf-lookup.c
+
+libctf_la-ctf-open.lo: ctf-open.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-open.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-open.Tpo -c -o libctf_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-open.Tpo $(DEPDIR)/libctf_la-ctf-open.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-open.c' object='libctf_la-ctf-open.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
+
+libctf_la-ctf-string.lo: ctf-string.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-string.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-string.Tpo -c -o libctf_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-string.Tpo $(DEPDIR)/libctf_la-ctf-string.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-string.c' object='libctf_la-ctf-string.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
+
+libctf_la-ctf-subr.lo: ctf-subr.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-subr.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-subr.Tpo -c -o libctf_la-ctf-subr.lo `test -f 'ctf-subr.c' || echo '$(srcdir)/'`ctf-subr.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-subr.Tpo $(DEPDIR)/libctf_la-ctf-subr.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-subr.c' object='libctf_la-ctf-subr.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-subr.lo `test -f 'ctf-subr.c' || echo '$(srcdir)/'`ctf-subr.c
+
+libctf_la-ctf-types.lo: ctf-types.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-types.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-types.Tpo -c -o libctf_la-ctf-types.lo `test -f 'ctf-types.c' || echo '$(srcdir)/'`ctf-types.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-types.Tpo $(DEPDIR)/libctf_la-ctf-types.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-types.c' object='libctf_la-ctf-types.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-types.lo `test -f 'ctf-types.c' || echo '$(srcdir)/'`ctf-types.c
+
+libctf_la-ctf-util.lo: ctf-util.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-util.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-util.Tpo -c -o libctf_la-ctf-util.lo `test -f 'ctf-util.c' || echo '$(srcdir)/'`ctf-util.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-util.Tpo $(DEPDIR)/libctf_la-ctf-util.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-util.c' object='libctf_la-ctf-util.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-util.lo `test -f 'ctf-util.c' || echo '$(srcdir)/'`ctf-util.c
+
+libctf_la-ctf-qsort_r.lo: ctf-qsort_r.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-qsort_r.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-qsort_r.Tpo -c -o libctf_la-ctf-qsort_r.lo `test -f 'ctf-qsort_r.c' || echo '$(srcdir)/'`ctf-qsort_r.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-qsort_r.Tpo $(DEPDIR)/libctf_la-ctf-qsort_r.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-qsort_r.c' object='libctf_la-ctf-qsort_r.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-qsort_r.lo `test -f 'ctf-qsort_r.c' || echo '$(srcdir)/'`ctf-qsort_r.c
+
+libctf_la-ctf-open-bfd.lo: ctf-open-bfd.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-open-bfd.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-open-bfd.Tpo -c -o libctf_la-ctf-open-bfd.lo `test -f 'ctf-open-bfd.c' || echo '$(srcdir)/'`ctf-open-bfd.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-open-bfd.Tpo $(DEPDIR)/libctf_la-ctf-open-bfd.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-open-bfd.c' object='libctf_la-ctf-open-bfd.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-open-bfd.lo `test -f 'ctf-open-bfd.c' || echo '$(srcdir)/'`ctf-open-bfd.c
+
 mostlyclean-libtool:
 	-rm -f *.lo
 
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index a964c20b9ed..85fd060262e 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -542,6 +542,7 @@ ctf_serialize (ctf_file_t *fp)
   nfp->ctf_link_type_mapping = fp->ctf_link_type_mapping;
   nfp->ctf_link_memb_name_changer = fp->ctf_link_memb_name_changer;
   nfp->ctf_link_memb_name_changer_arg = fp->ctf_link_memb_name_changer_arg;
+  nfp->ctf_link_flags = fp->ctf_link_flags;
 
   nfp->ctf_snapshot_lu = fp->ctf_snapshots;
 
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 71b732a2775..cb7de23dbbb 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -314,6 +314,9 @@ struct ctf_file
   ctf_dynhash_t *ctf_link_type_mapping; /* Map input types to output types.  */
   ctf_dynhash_t *ctf_link_cu_mapping;	/* Map CU names to CTF dict names.  */
   /* Allow the caller to Change the name of link archive members.  */
+  /* CTF linker flags.  */
+  int ctf_link_flags;
+
   ctf_link_memb_name_changer_f *ctf_link_memb_name_changer;
   void *ctf_link_memb_name_changer_arg; /* Argument for it.  */
   ctf_dynhash_t *ctf_add_processing; /* Types ctf_add_type is working on now.  */
@@ -560,6 +563,7 @@ _libctf_printflike_ (3, 4)
 extern void ctf_err_warn (ctf_file_t *, int is_warning, const char *, ...);
 extern void ctf_assert_fail_internal (ctf_file_t *, const char *,
 				      size_t, const char *);
+extern const char *ctf_link_input_name (ctf_file_t *);
 
 extern Elf64_Sym *ctf_sym_to_elf64 (const Elf32_Sym *src, Elf64_Sym *dst);
 extern const char *ctf_lookup_symbol_name (ctf_file_t *fp, unsigned long symidx);
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index bcfd2564fbc..705701d0eb5 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -20,6 +20,10 @@
 #include <ctf-impl.h>
 #include <string.h>
 
+#if defined (PIC)
+#pragma weak ctf_open
+#endif
+
 /* Type tracking machinery.  */
 
 /* Record the correspondence between a source and ctf_add_type()-added
@@ -137,45 +141,143 @@ ctf_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type, ctf_file_t **dst_fp)
    (otherwise) and returns it, suitable for addition in the .ctf section of the
    output.  */
 
-/* Add a file to a link.  */
-
-static void ctf_arc_close_thunk (void *arc)
+/* Return the name of the compilation unit this CTF dict or its parent applies
+   to, or a non-null string otherwise: prefer the parent.  Used in debugging
+   output.  Sometimes used for outputs too.  */
+const char *
+ctf_link_input_name (ctf_file_t *fp)
 {
-  ctf_arc_close ((ctf_archive_t *) arc);
+  if (fp->ctf_parent && fp->ctf_parent->ctf_cuname)
+    return fp->ctf_parent->ctf_cuname;
+  else if (fp->ctf_cuname)
+    return fp->ctf_cuname;
+  else
+    return "(unnamed)";
 }
 
-static void ctf_file_close_thunk (void *file)
+/* The linker inputs look like this.  clin_fp is used for short-circuited
+   CU-mapped links that can entirely avoid the first link phase in some
+   situations in favour of just passing on the contained ctf_file_t: it is
+   always the sole ctf_file_t inside the corresponding clin_arc.  If set, it
+   gets assigned directly to the final link inputs and freed from there, so it
+   never gets explicitly freed in the ctf_link_input.  */
+typedef struct ctf_link_input
+{
+  const char *clin_filename;
+  ctf_archive_t *clin_arc;
+  ctf_file_t *clin_fp;
+  int n;
+} ctf_link_input_t;
+
+static void
+ctf_link_input_close (void *input)
 {
-  ctf_file_close ((ctf_file_t *) file);
+  ctf_link_input_t *i = (ctf_link_input_t *) input;
+  if (i->clin_arc)
+    ctf_arc_close (i->clin_arc);
+  free (i);
 }
 
-int
-ctf_link_add_ctf (ctf_file_t *fp, ctf_archive_t *ctf, const char *name)
+/* Like ctf_link_add_ctf, below, but with no error-checking, so it can be called
+   in the middle of an ongoing link.  */
+static int
+ctf_link_add_ctf_internal (ctf_file_t *fp, ctf_archive_t *ctf,
+			   ctf_file_t *fp_input, const char *name)
 {
+  ctf_link_input_t *input = NULL;
   char *dupname = NULL;
 
-  if (fp->ctf_link_outputs)
-    return (ctf_set_errno (fp, ECTF_LINKADDEDLATE));
-  if (fp->ctf_link_inputs == NULL)
-    fp->ctf_link_inputs = ctf_dynhash_create (ctf_hash_string,
-					      ctf_hash_eq_string, free,
-					      ctf_arc_close_thunk);
-
-  if (fp->ctf_link_inputs == NULL)
+  if ((input = calloc (1, sizeof (ctf_link_input_t))) == NULL)
     goto oom;
 
   if ((dupname = strdup (name)) == NULL)
     goto oom;
 
-  if (ctf_dynhash_insert (fp->ctf_link_inputs, dupname, ctf) < 0)
+  input->clin_arc = ctf;
+  input->clin_fp = fp_input;
+  input->clin_filename = dupname;
+  input->n = ctf_dynhash_elements (fp->ctf_link_inputs);
+
+  if (ctf_dynhash_insert (fp->ctf_link_inputs, dupname, input) < 0)
     goto oom;
 
   return 0;
  oom:
-  free (fp->ctf_link_inputs);
-  fp->ctf_link_inputs = NULL;
+  free (input);
   free (dupname);
-  return (ctf_set_errno (fp, ENOMEM));
+  return ctf_set_errno (fp, ENOMEM);
+}
+
+/* Add a file, memory buffer, or unopened file (by name) to a link.
+
+   You can call this with:
+
+    CTF and NAME: link the passed ctf_archive_t, with the given NAME.
+    NAME alone: open NAME as a CTF file when needed.
+    BUF and NAME: open the BUF (of length N) as CTF, with the given NAME.  (Not
+    yet implemented.)
+
+    Passed in CTF args are owned by the dictionary and will be freed by it.
+    The BUF arg is *not* owned by the dictionary, and the user should not free
+    its referent until the link is done.
+
+    The order of calls to this function influences the order of types in the
+    final link output, but otherwise is not important.
+
+    Private for now, but may in time become public once support for BUF is
+    implemented.  */
+
+static int
+ctf_link_add (ctf_file_t *fp, ctf_archive_t *ctf, const char *name,
+	      void *buf _libctf_unused_, size_t n _libctf_unused_)
+{
+  if (buf)
+    return (ctf_set_errno (fp, ECTF_NOTYET));
+
+  if (!((ctf && name && !buf)
+	|| (name && !buf && !ctf)
+	|| (buf && name && !ctf)))
+    return (ctf_set_errno (fp, EINVAL));
+
+  /* We can only lazily open files if libctf.so is in use rather than
+     libctf-nobfd.so.  This is a little tricky: in shared libraries, we can use
+     a weak symbol so that -lctf -lctf-nobfd works, but in static libraries we
+     must distinguish between the two libraries explicitly.  */
+
+#if defined (PIC)
+  if (!buf && !ctf && name && !ctf_open)
+    return (ctf_set_errno (fp, ECTF_NEEDSBFD));
+#elif NOBFD
+  if (!buf && !ctf && name)
+    return (ctf_set_errno (fp, ECTF_NEEDSBFD));
+#endif
+
+  if (fp->ctf_link_outputs)
+    return (ctf_set_errno (fp, ECTF_LINKADDEDLATE));
+  if (fp->ctf_link_inputs == NULL)
+    fp->ctf_link_inputs = ctf_dynhash_create (ctf_hash_string,
+					      ctf_hash_eq_string, free,
+					      ctf_link_input_close);
+
+  if (fp->ctf_link_inputs == NULL)
+    return (ctf_set_errno (fp, ENOMEM));
+
+  return ctf_link_add_ctf_internal (fp, ctf, NULL, name);
+}
+
+/* Add an opened CTF archive or unopened file (by name) to a link.
+   If CTF is NULL and NAME is non-null, an unopened file is meant:
+   otherwise, the specified archive is assumed to have the given NAME.
+
+    Passed in CTF args are owned by the dictionary and will be freed by it.
+
+    The order of calls to this function influences the order of types in the
+    final link output, but otherwise is not important.  */
+
+int
+ctf_link_add_ctf (ctf_file_t *fp, ctf_archive_t *ctf, const char *name)
+{
+  return ctf_link_add (fp, ctf, name, NULL, 0);
 }
 
 /* Return a per-CU output CTF dictionary suitable for the given CU, creating and
@@ -210,9 +312,9 @@ ctf_create_per_cu (ctf_file_t *fp, const char *filename, const char *cuname)
 
       if ((cu_fp = ctf_create (&err)) == NULL)
 	{
-	  ctf_dprintf ("Cannot create per-CU CTF archive for CU %s from "
-		       "input file %s: %s\n", cuname, filename,
-		       ctf_errmsg (err));
+	  ctf_err_warn (fp, 0, "Cannot create per-CU CTF archive for "
+			"CU %s from input file %s: %s", cuname, filename,
+			ctf_errmsg (err));
 	  ctf_set_errno (fp, err);
 	  return NULL;
 	}
@@ -262,7 +364,8 @@ ctf_link_add_cu_mapping (ctf_file_t *fp, const char *from, const char *to)
   if (fp->ctf_link_outputs == NULL)
     fp->ctf_link_outputs = ctf_dynhash_create (ctf_hash_string,
 					       ctf_hash_eq_string, free,
-					       ctf_file_close_thunk);
+					       (ctf_hash_free_fun)
+					       ctf_file_close);
 
   if (fp->ctf_link_outputs == NULL)
     return ctf_set_errno (fp, ENOMEM);
@@ -310,15 +413,27 @@ ctf_link_set_memb_name_changer (ctf_file_t *fp,
 
 typedef struct ctf_link_in_member_cb_arg
 {
+  /* The shared output dictionary.  */
   ctf_file_t *out_fp;
-  const char *file_name;
+
+  /* The filename of the input file, and an fp to each dictionary in that file
+     in turn.  */
+  const char *in_file_name;
   ctf_file_t *in_fp;
-  ctf_file_t *main_input_fp;
+
+  /* The CU name of the dict being processed.  */
   const char *cu_name;
-  char *arcname;
-  int done_main_member;
-  int share_mode;
   int in_input_cu_file;
+
+  /* The parent dictionary in the input, and whether it's been processed yet.
+     Not needed by ctf_link_one_type / ctf_link_one_variable, only by higher
+     layers.  */
+  ctf_file_t *in_fp_parent;
+  int done_parent;
+
+  /* If true, this is the CU-mapped portion of a deduplicating link: no child
+     dictionaries should be created.  */
+  int cu_mapped;
 } ctf_link_in_member_cb_arg_t;
 
 /* Link one type into the link.  We rely on ctf_add_type() to detect
@@ -332,9 +447,9 @@ ctf_link_one_type (ctf_id_t type, int isroot _libctf_unused_, void *arg_)
   ctf_file_t *per_cu_out_fp;
   int err;
 
-  if (arg->share_mode != CTF_LINK_SHARE_UNCONFLICTED)
+  if (arg->in_fp->ctf_link_flags != CTF_LINK_SHARE_UNCONFLICTED)
     {
-      ctf_dprintf ("Share-duplicated mode not yet implemented.\n");
+      ctf_err_warn (arg->out_fp, 0, "Share-duplicated mode not yet implemented");
       return ctf_set_errno (arg->out_fp, ECTF_NOTYET);
     }
 
@@ -352,19 +467,18 @@ ctf_link_one_type (ctf_id_t type, int isroot _libctf_unused_, void *arg_)
       if (err != ECTF_CONFLICT)
 	{
 	  if (err != ECTF_NONREPRESENTABLE)
-	    ctf_dprintf ("Cannot link type %lx from archive member %s, input file %s "
-			 "into output link: %s\n", type, arg->arcname, arg->file_name,
-			 ctf_errmsg (err));
+	    ctf_err_warn (arg->out_fp, 1, "Cannot link type %lx from input file %s, "
+			  "CU %s into output link: %s", type, arg->cu_name,
+			 arg->in_file_name, ctf_errmsg (err));
 	  /* We must ignore this problem or we end up losing future types, then
 	     trying to link the variables in, then exploding.  Better to link as
-	     much as possible.  XXX when we add a proper link warning
-	     infrastructure, we should report the error here!  */
+	     much as possible.  */
 	  return 0;
 	}
       ctf_set_errno (arg->out_fp, 0);
     }
 
-  if ((per_cu_out_fp = ctf_create_per_cu (arg->out_fp, arg->file_name,
+  if ((per_cu_out_fp = ctf_create_per_cu (arg->out_fp, arg->in_file_name,
 					  arg->cu_name)) == NULL)
     return -1;					/* Errno is set for us.  */
 
@@ -373,10 +487,10 @@ ctf_link_one_type (ctf_id_t type, int isroot _libctf_unused_, void *arg_)
 
   err = ctf_errno (per_cu_out_fp);
   if (err != ECTF_NONREPRESENTABLE)
-    ctf_dprintf ("Cannot link type %lx from CTF archive member %s, input file %s "
-		 "into output per-CU CTF archive member %s: %s: skipped\n", type,
-		 arg->arcname, arg->file_name, arg->arcname,
-		 ctf_errmsg (err));
+    ctf_err_warn (arg->out_fp, 1, "Cannot link type %lx from input file %s, CU %s "
+		 "into output per-CU CTF archive member %s: %s: skipped", type,
+		 ctf_link_input_name (arg->in_fp), arg->in_file_name,
+		 ctf_link_input_name (per_cu_out_fp), ctf_errmsg (err));
   if (err == ECTF_CONFLICT)
       /* Conflicts are possible at this stage only if a non-ld user has combined
 	 multiple TUs into a single output dictionary.  Even in this case we do not
@@ -402,8 +516,9 @@ check_variable (const char *name, ctf_file_t *fp, ctf_id_t type,
   if (dvd->dvd_type != type)
     {
       /* Variable here.  Wrong type: cannot add.  Just skip it, because there is
-	 no way to express this in CTF.  (This might be the parent, in which
-	 case we'll try adding in the child first, and only then give up.)  */
+	 no way to express this in CTF.  Don't even warn: this case is too
+	 common.  (This might be the parent, in which case we'll try adding in
+	 the child first, and only then give up.)  */
       ctf_dprintf ("Inexpressible duplicate variable %s skipped.\n", name);
     }
 
@@ -448,9 +563,18 @@ ctf_link_one_variable (const char *name, ctf_id_t type, void *arg_)
 
   /* Can't add to the parent due to a name clash, or because it references a
      type only present in the child.  Try adding to the child, creating if need
-     be.  */
+     be.  If we can't do that, skip it.  Don't add to a child if we're doing a
+     CU-mapped link, since that has only one output.  */
+
+  if (arg->cu_mapped)
+    {
+      ctf_dprintf ("Variable %s in input file %s depends on a type %lx hidden "
+		   "due to conflicts: skipped.\n", name, arg->in_file_name,
+		   type);
+      return 0;
+    }
 
-  if ((per_cu_out_fp = ctf_create_per_cu (arg->out_fp, arg->file_name,
+  if ((per_cu_out_fp = ctf_create_per_cu (arg->out_fp, arg->in_file_name,
 					  arg->cu_name)) == NULL)
     return -1;					/* Errno is set for us.  */
 
@@ -462,8 +586,9 @@ ctf_link_one_variable (const char *name, ctf_id_t type, void *arg_)
 
       if (dst_type == 0)
 	{
-	  ctf_dprintf ("Type %lx for variable %s in input file %s not "
-		       "found: skipped.\n", type, name, arg->file_name);
+	  ctf_err_warn (arg->out_fp, 1, "Type %lx for variable %s in input "
+			"file %s not found: skipped", type, name,
+			arg->in_file_name);
 	  /* Do not terminate the link: just skip the variable.  */
 	  return 0;
 	}
@@ -475,11 +600,11 @@ ctf_link_one_variable (const char *name, ctf_id_t type, void *arg_)
   return 0;
 }
 
-/* Merge every type and variable in this archive member into the link, so we can
-   relink things that have already had ld run on them.  We use the archive
-   member name, sans any leading '.ctf.', as the CU name for ambiguous types if
-   there is one and it's not the default: otherwise, we use the name of the
-   input file.  */
+/* Merge every type (and optionally, variable) in this archive member into the
+   link, so we can relink things that have already had ld run on them.  We use
+   the archive member name, sans any leading '.ctf.', as the CU name for
+   ambiguous types if there is one and it's not the default: otherwise, we use
+   the name of the input file.  */
 static int
 ctf_link_one_input_archive_member (ctf_file_t *in_fp, const char *name, void *arg_)
 {
@@ -497,32 +622,16 @@ ctf_link_one_input_archive_member (ctf_file_t *in_fp, const char *name, void *ar
 	 causes the system to erroneously conclude that all types are duplicated
 	 and should be shared, even if they are not.  */
 
-      if (arg->done_main_member)
+      if (arg->done_parent)
 	return 0;
-      arg->arcname = strdup (".ctf.");
-      if (arg->arcname)
-	{
-	  char *new_name;
-
-	  new_name = ctf_str_append (arg->arcname, arg->file_name);
-	  if (new_name)
-	    arg->arcname = new_name;
-	  else
-	    free (arg->arcname);
-	}
     }
   else
     {
-      arg->arcname = strdup (name);
-
       /* Get ambiguous types from our parent.  */
-      ctf_import (in_fp, arg->main_input_fp);
+      ctf_import (in_fp, arg->in_fp_parent);
       arg->in_input_cu_file = 1;
     }
 
-  if (!arg->arcname)
-    return ctf_set_errno (in_fp, ENOMEM);
-
   arg->cu_name = name;
   if (strncmp (arg->cu_name, ".ctf.", strlen (".ctf.")) == 0)
     arg->cu_name += strlen (".ctf.");
@@ -532,7 +641,6 @@ ctf_link_one_input_archive_member (ctf_file_t *in_fp, const char *name, void *ar
     err = ctf_variable_iter (in_fp, ctf_link_one_variable, arg);
 
   arg->in_input_cu_file = 0;
-  free (arg->arcname);
 
   if (err < 0)
       return -1;				/* Errno is set for us.  */
@@ -551,61 +659,128 @@ empty_link_type_mapping (void *key _libctf_unused_, void *value,
     ctf_dynhash_empty (fp->ctf_link_type_mapping);
 }
 
+/* Lazily open a CTF archive for linking, if not already open.
+
+   Returns the number of files contained within the opened archive (0 for none),
+   or -1 on error, as usual.  */
+static ssize_t
+ctf_link_lazy_open (ctf_file_t *fp, ctf_link_input_t *input)
+{
+  size_t count;
+  int err;
+
+  if (input->clin_arc)
+    return ctf_archive_count (input->clin_arc);
+
+  if (input->clin_fp)
+    return 1;
+
+  /* See ctf_link_add_ctf.  */
+#if defined (PIC) || !NOBFD
+  input->clin_arc = ctf_open (input->clin_filename, NULL, &err);
+#else
+  ctf_err_warn (fp, 0, "Cannot open %s lazily: %s", input->clin_filename,
+		ctf_errmsg (ECTF_NEEDSBFD));
+  ctf_set_errno (fp, ECTF_NEEDSBFD);
+  return -1;
+#endif
+
+  /* Having no CTF sections is not an error.  We just don't need to do
+     anything.  */
+
+  if (!input->clin_arc)
+    {
+      if (err == ECTF_NOCTFDATA)
+	return 0;
+
+      ctf_err_warn (fp, 0, "Opening CTF %s failed: %s",
+		    input->clin_filename, ctf_errmsg (err));
+      ctf_set_errno (fp, err);
+      return -1;
+    }
+
+  if ((count = ctf_archive_count (input->clin_arc)) == 0)
+    ctf_arc_close (input->clin_arc);
+
+  return (ssize_t) count;
+}
+
+/* Close an input, as a ctf_dynhash_iter iterator.  */
+static void
+ctf_link_close_one_input_archive (void *key _libctf_unused_, void *value,
+				  void *arg _libctf_unused_)
+{
+  ctf_link_input_t *input = (ctf_link_input_t *) value;
+  if (input->clin_arc)
+    ctf_arc_close (input->clin_arc);
+  input->clin_arc = NULL;
+}
+
 /* Link one input file's types into the output file.  */
 static void
 ctf_link_one_input_archive (void *key, void *value, void *arg_)
 {
   const char *file_name = (const char *) key;
-  ctf_archive_t *arc = (ctf_archive_t *) value;
+  ctf_link_input_t *input = (ctf_link_input_t *)value;
   ctf_link_in_member_cb_arg_t *arg = (ctf_link_in_member_cb_arg_t *) arg_;
-  int err;
+  int err = 0;
+
+  if (!input->clin_arc)
+    {
+      err = ctf_link_lazy_open (arg->out_fp, input);
+      if (err == 0)				/* Just no CTF.  */
+	return;
+
+      if (err < 0)
+	return;					/* errno is set for us.  */
+    }
 
-  arg->file_name = file_name;
-  arg->done_main_member = 0;
-  if ((arg->main_input_fp = ctf_arc_open_by_name (arc, NULL, &err)) == NULL)
+  arg->in_file_name = file_name;
+  arg->done_parent = 0;
+  if ((arg->in_fp_parent = ctf_arc_open_by_name (input->clin_arc, NULL,
+						 &err)) == NULL)
     if (err != ECTF_ARNNAME)
       {
-	ctf_dprintf ("Cannot open main archive member in input file %s in the "
-		     "link: skipping: %s.\n", arg->file_name,
-		     ctf_errmsg (err));
-	return;
+	ctf_err_warn (arg->out_fp, 0, "Cannot open main archive member in "
+		      "input file %s in the link: skipping: %s",
+		      arg->in_file_name, ctf_errmsg (err));
+	goto out;
       }
 
-  if (ctf_link_one_input_archive_member (arg->main_input_fp,
+  if (ctf_link_one_input_archive_member (arg->in_fp_parent,
 					 _CTF_SECTION, arg) < 0)
     {
-      ctf_file_close (arg->main_input_fp);
-      return;
+      ctf_file_close (arg->in_fp_parent);
+      goto out;
     }
-  arg->done_main_member = 1;
-  if (ctf_archive_iter (arc, ctf_link_one_input_archive_member, arg) < 0)
-    ctf_dprintf ("Cannot traverse archive in input file %s: link "
-		 "cannot continue: %s.\n", arg->file_name,
-		 ctf_errmsg (ctf_errno (arg->out_fp)));
+  arg->done_parent = 1;
+  if (ctf_archive_iter (input->clin_arc, ctf_link_one_input_archive_member,
+			arg) < 0)
+    ctf_err_warn (arg->out_fp, 0, "Cannot traverse archive in input file %s: "
+		  "link cannot continue: %s", arg->in_file_name,
+		  ctf_errmsg (ctf_errno (arg->out_fp)));
   else
     {
       /* The only error indication to the caller is the errno: so ensure that it
 	 is zero if there was no actual error from the caller.  */
       ctf_set_errno (arg->out_fp, 0);
     }
-  ctf_file_close (arg->main_input_fp);
+  ctf_file_close (arg->in_fp_parent);
 
-  /* Discard the now-unnecessary mapping table data.  */
-  if (arg->out_fp->ctf_link_type_mapping)
-    ctf_dynhash_empty (arg->out_fp->ctf_link_type_mapping);
-  ctf_dynhash_iter (arg->out_fp->ctf_link_outputs, empty_link_type_mapping, NULL);
+ out:
+  ctf_link_close_one_input_archive (key, value, NULL);
 }
 
 /* Merge types and variable sections in all files added to the link
-   together.  */
+   together.  All the added files are closed.  */
 int
-ctf_link (ctf_file_t *fp, int share_mode)
+ctf_link (ctf_file_t *fp, int flags)
 {
   ctf_link_in_member_cb_arg_t arg;
 
   memset (&arg, 0, sizeof (struct ctf_link_in_member_cb_arg));
   arg.out_fp = fp;
-  arg.share_mode = share_mode;
+  fp->ctf_link_flags = flags;
 
   if (fp->ctf_link_inputs == NULL)
     return 0;					/* Nothing to do. */
@@ -613,7 +788,8 @@ ctf_link (ctf_file_t *fp, int share_mode)
   if (fp->ctf_link_outputs == NULL)
     fp->ctf_link_outputs = ctf_dynhash_create (ctf_hash_string,
 					       ctf_hash_eq_string, free,
-					       ctf_file_close_thunk);
+					       (ctf_hash_free_fun)
+					       ctf_file_close);
 
   if (fp->ctf_link_outputs == NULL)
     return ctf_set_errno (fp, ENOMEM);
@@ -621,7 +797,12 @@ ctf_link (ctf_file_t *fp, int share_mode)
   ctf_dynhash_iter (fp->ctf_link_inputs, ctf_link_one_input_archive,
 		    &arg);
 
-  if (ctf_errno (fp) != 0)
+  /* Discard the now-unnecessary mapping table data from all the outputs.  */
+  if (fp->ctf_link_type_mapping)
+    ctf_dynhash_empty (fp->ctf_link_type_mapping);
+  ctf_dynhash_iter (fp->ctf_link_outputs, empty_link_type_mapping, NULL);
+
+  if ((ctf_errno (fp) != 0) && (ctf_errno (fp) != ECTF_NOCTFDATA))
     return -1;
   return 0;
 }
@@ -908,7 +1089,7 @@ ctf_link_write (ctf_file_t *fp, size_t *size, size_t threshold)
 	free (arg.dynames[i]);
       free (arg.dynames);
     }
-  ctf_dprintf ("Cannot write archive in link: %s failure: %s\n", errloc,
-	       ctf_errmsg (ctf_errno (fp)));
+  ctf_err_warn (fp, 0, "Cannot write archive in link: %s failure: %s", errloc,
+		ctf_errmsg (ctf_errno (fp)));
   return NULL;
 }
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index f1c9b2bf003..7eed14ae54d 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -156,8 +156,6 @@ LIBCTF_1.0 {
 	ctf_getdebug;
 	ctf_errwarning_next;
 
-	/* Not yet part of the stable API.  */
-
 	ctf_link_add_ctf;
 	ctf_link_add_cu_mapping;
 	ctf_link_set_memb_name_changer;
-- 
2.27.0.247.g3dff7de930

[PATCH 44/59] libctf, link: fix ctf_link_write fd leak

[Nick Alcock]

We were leaking the fd on every invocation.

libctf/
	* ctf-link.c (ctf_link_write): Close the fd.
---
 libctf/ctf-link.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 705701d0eb5..fa15c9bf9bb 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -1071,6 +1071,7 @@ ctf_link_write (ctf_file_t *fp, size_t *size, size_t threshold)
 	free (arg.dynames[i]);
       free (arg.dynames);
     }
+  fclose (f);
   return buf;
 
  err_no:
-- 
2.27.0.247.g3dff7de930

[PATCH 45/59] libctf, link: redo cu-mapping handling

[Nick Alcock]

Now a bunch of stuff that doesn't apply to ld or any normal use of
libctf, piled into one commit so that it's easier to ignore.

The cu-mapping machinery associates incoming compilation unit names with
outgoing names of CTF dictionaries that should correspond to them, for
non-gdb CTF consumers that would like to group multiple TUs into a
single child dict if conflicting types are found in it (the existing use
case is one kernel module, one child CTF dict, even if the kernel module
is composed of multiple CUs).

The upcoming deduplicator needs to track not only the mapping from
incoming CU name to outgoing dict name, but the inverse mapping from
outgoing dict name to incoming CU name, so it can work over every CTF
dict we might see in the output and link into it.

So rejig the ctf-link machinery to do that.  Simultaneously (because
they are closely associated and were written at the same time), we add a
new CTF_LINK_EMPTY_CU_MAPPINGS flag to ctf_link, which tells the
ctf_link machinery to create empty child dicts for each outgoing CU
mapping even if no CUs that correspond to it exist in the link.  This is
a bit (OK, quite a lot) of a waste of space, but some existing consumers
require it.  (Nobody else should use it.)

Its value is not consecutive with existing CTF_LINK flag values because
we're about to add more flags that are conceptually closer to the
existing ones than this one is.

include/
	* ctf-api.h (CTF_LINK_EMPTY_CU_MAPPINGS): New.

libctf/
	* ctf-impl.h (ctf_file_t): Improve comments.
	<ctf_link_cu_mapping>: Split into...
	<ctf_link_in_cu_mapping>: ... this...
	<ctf_link_out_cu_mapping>: ... and this.
	* ctf-create.c (ctf_serialize): Adjust.
	* ctf-open.c (ctf_file_close): Likewise.
	* ctf-link.c (ctf_create_per_cu): Look things up in the
	in_cu_mapping instead of the cu_mapping.
	(ctf_link_add_cu_mapping): The deduplicating link will define
	what happens if many FROMs share a TO.
	(ctf_link_add_cu_mapping): Create in_cu_mapping and
	out_cu_mapping. Do not create ctf_link_outputs here any more, or
	create per-CU dicts here: they are already created when needed.
	(ctf_link_one_variable): Log a debug message if we skip a
	variable due to its type being concealed in a CU-mapped link.
	(This is probably too common a case to make into a warning.)
	(ctf_link): Create empty per-CU dicts if requested.
---
 include/ctf-api.h   |   6 ++-
 libctf/ctf-create.c |   6 ++-
 libctf/ctf-impl.h   |  21 +++++++--
 libctf/ctf-link.c   | 107 +++++++++++++++++++++++++++++++++-----------
 libctf/ctf-open.c   |   3 +-
 5 files changed, 111 insertions(+), 32 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 7d3e1c8bfcd..700a2b1ef5d 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -78,7 +78,7 @@ typedef struct ctf_link_sym
   uint32_t st_value;
 } ctf_link_sym_t;
 
-/* Indication of how to share types when linking.  */
+/* Flags applying to this specific link.  */
 
 /* Share all types that are not in conflict.  The default.  */
 #define CTF_LINK_SHARE_UNCONFLICTED 0x0
@@ -86,6 +86,10 @@ typedef struct ctf_link_sym
 /* Share only types that are used by multiple inputs.  Not implemented yet.  */
 #define CTF_LINK_SHARE_DUPLICATED 0x1
 
+/* Create empty outputs for all registered CU mappings even if no types are
+   emitted into them.  */
+#define CTF_LINK_EMPTY_CU_MAPPINGS 0x4
+
 /* Symbolic names for CTF sections.  */
 
 typedef enum ctf_sect_names
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 85fd060262e..10c6bbf552e 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -538,7 +538,8 @@ ctf_serialize (ctf_file_t *fp)
   nfp->ctf_errs_warnings = fp->ctf_errs_warnings;
   nfp->ctf_str_prov_offset = fp->ctf_str_prov_offset;
   nfp->ctf_syn_ext_strtab = fp->ctf_syn_ext_strtab;
-  nfp->ctf_link_cu_mapping = fp->ctf_link_cu_mapping;
+  nfp->ctf_link_in_cu_mapping = fp->ctf_link_in_cu_mapping;
+  nfp->ctf_link_out_cu_mapping = fp->ctf_link_out_cu_mapping;
   nfp->ctf_link_type_mapping = fp->ctf_link_type_mapping;
   nfp->ctf_link_memb_name_changer = fp->ctf_link_memb_name_changer;
   nfp->ctf_link_memb_name_changer_arg = fp->ctf_link_memb_name_changer_arg;
@@ -565,7 +566,8 @@ ctf_serialize (ctf_file_t *fp)
   fp->ctf_link_inputs = NULL;
   fp->ctf_link_outputs = NULL;
   fp->ctf_syn_ext_strtab = NULL;
-  fp->ctf_link_cu_mapping = NULL;
+  fp->ctf_link_in_cu_mapping = NULL;
+  fp->ctf_link_out_cu_mapping = NULL;
   fp->ctf_link_type_mapping = NULL;
   fp->ctf_parent_unreffed = 1;
 
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index cb7de23dbbb..46bceb49861 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -311,12 +311,27 @@ struct ctf_file
   ctf_list_t ctf_errs_warnings;	  /* CTF errors and warnings.  */
   ctf_dynhash_t *ctf_link_inputs; /* Inputs to this link.  */
   ctf_dynhash_t *ctf_link_outputs; /* Additional outputs from this link.  */
-  ctf_dynhash_t *ctf_link_type_mapping; /* Map input types to output types.  */
-  ctf_dynhash_t *ctf_link_cu_mapping;	/* Map CU names to CTF dict names.  */
-  /* Allow the caller to Change the name of link archive members.  */
+
+  /* Map input types to output types: populated in each output dict.
+     Key is a ctf_link_type_key_t: value is a type ID.  Used by
+     nondeduplicating links and ad-hoc ctf_add_type calls only.  */
+  ctf_dynhash_t *ctf_link_type_mapping;
+
+  /* Map input CU names to output CTF dict names: populated in the top-level
+     output dict.
+
+     Key and value are dynamically-allocated strings.  */
+  ctf_dynhash_t *ctf_link_in_cu_mapping;
+
+  /* Map output CTF dict names to input CU names: populated in the top-level
+     output dict.  A hash of string to hash (set) of strings.  Key and
+     individual value members are shared with ctf_link_in_cu_mapping.  */
+  ctf_dynhash_t *ctf_link_out_cu_mapping;
+
   /* CTF linker flags.  */
   int ctf_link_flags;
 
+  /* Allow the caller to change the name of link archive members.  */
   ctf_link_memb_name_changer_f *ctf_link_memb_name_changer;
   void *ctf_link_memb_name_changer_arg; /* Argument for it.  */
   ctf_dynhash_t *ctf_add_processing; /* Types ctf_add_type is working on now.  */
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index fa15c9bf9bb..3c96604b36a 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -296,10 +296,12 @@ ctf_create_per_cu (ctf_file_t *fp, const char *filename, const char *cuname)
      dictionary name.  We prefer the filename because this is easier for likely
      callers to determine.  */
 
-  if (fp->ctf_link_cu_mapping)
+  if (fp->ctf_link_in_cu_mapping)
     {
-      if (((ctf_name = ctf_dynhash_lookup (fp->ctf_link_cu_mapping, filename)) == NULL) &&
-	  ((ctf_name = ctf_dynhash_lookup (fp->ctf_link_cu_mapping, cuname)) == NULL))
+      if (((ctf_name = ctf_dynhash_lookup (fp->ctf_link_in_cu_mapping,
+					   filename)) == NULL) &&
+	  ((ctf_name = ctf_dynhash_lookup (fp->ctf_link_in_cu_mapping,
+					   cuname)) == NULL))
 	ctf_name = filename;
     }
 
@@ -339,10 +341,7 @@ ctf_create_per_cu (ctf_file_t *fp, const char *filename, const char *cuname)
 
 /* Add a mapping directing that the CU named FROM should have its
    conflicting/non-duplicate types (depending on link mode) go into a container
-   named TO.  Many FROMs can share a TO: in this case, the effect on conflicting
-   types is not yet defined (but in time an auto-renaming algorithm will be
-   added: ugly, but there is really no right thing one can do in this
-   situation).
+   named TO.  Many FROMs can share a TO.
 
    We forcibly add a container named TO in every case, even though it may well
    wind up empty, because clients that use this facility usually expect to find
@@ -352,34 +351,63 @@ int
 ctf_link_add_cu_mapping (ctf_file_t *fp, const char *from, const char *to)
 {
   int err;
-  char *f, *t;
+  char *f = NULL, *t = NULL;
+  ctf_dynhash_t *one_out;
+
+  if (fp->ctf_link_in_cu_mapping == NULL)
+    fp->ctf_link_in_cu_mapping = ctf_dynhash_create (ctf_hash_string,
+						     ctf_hash_eq_string, free,
+						     free);
+  if (fp->ctf_link_in_cu_mapping == NULL)
+    goto oom;
 
-  if (fp->ctf_link_cu_mapping == NULL)
-    fp->ctf_link_cu_mapping = ctf_dynhash_create (ctf_hash_string,
-						  ctf_hash_eq_string, free,
-						  free);
-  if (fp->ctf_link_cu_mapping == NULL)
-    return ctf_set_errno (fp, ENOMEM);
+  if (fp->ctf_link_out_cu_mapping == NULL)
+    fp->ctf_link_out_cu_mapping = ctf_dynhash_create (ctf_hash_string,
+						      ctf_hash_eq_string, free,
+						      (ctf_hash_free_fun)
+						      ctf_dynhash_destroy);
+  if (fp->ctf_link_out_cu_mapping == NULL)
+    goto oom;
 
-  if (fp->ctf_link_outputs == NULL)
-    fp->ctf_link_outputs = ctf_dynhash_create (ctf_hash_string,
-					       ctf_hash_eq_string, free,
-					       (ctf_hash_free_fun)
-					       ctf_file_close);
+  f = strdup (from);
+  t = strdup (to);
+  if (!f || !t)
+    goto oom;
 
-  if (fp->ctf_link_outputs == NULL)
-    return ctf_set_errno (fp, ENOMEM);
+  /* Track both in a list from FROM to TO and in a list from TO to a list of
+     FROM.  The former is used to create TUs with the mapped-to name at need:
+     the latter is used in deduplicating links to pull in all input CUs
+     corresponding to a single output CU.  */
+
+  if ((err = ctf_dynhash_insert (fp->ctf_link_in_cu_mapping, f, t)) < 0)
+    {
+      ctf_set_errno (fp, err);
+      goto oom_noerrno;
+    }
 
+  /* f and t are now owned by the in_cu_mapping: reallocate them.  */
   f = strdup (from);
   t = strdup (to);
   if (!f || !t)
     goto oom;
 
-  if (ctf_create_per_cu (fp, t, t) == NULL)
-    goto oom_noerrno;				/* Errno is set for us.  */
+  if ((one_out = ctf_dynhash_lookup (fp->ctf_link_out_cu_mapping, t)) == NULL)
+    {
+      if ((one_out = ctf_dynhash_create (ctf_hash_string, ctf_hash_eq_string,
+					 free, NULL)) == NULL)
+	goto oom;
+      if ((err = ctf_dynhash_insert (fp->ctf_link_out_cu_mapping,
+				     t, one_out)) < 0)
+	{
+	  ctf_dynhash_destroy (one_out);
+	  ctf_set_errno (fp, err);
+	  goto oom_noerrno;
+	}
+    }
+  else
+    free (t);
 
-  err = ctf_dynhash_insert (fp->ctf_link_cu_mapping, f, t);
-  if (err)
+  if (ctf_dynhash_insert (one_out, f, NULL) < 0)
     {
       ctf_set_errno (fp, err);
       goto oom_noerrno;
@@ -777,6 +805,8 @@ int
 ctf_link (ctf_file_t *fp, int flags)
 {
   ctf_link_in_member_cb_arg_t arg;
+  ctf_next_t *i = NULL;
+  int err;
 
   memset (&arg, 0, sizeof (struct ctf_link_in_member_cb_arg));
   arg.out_fp = fp;
@@ -794,6 +824,33 @@ ctf_link (ctf_file_t *fp, int flags)
   if (fp->ctf_link_outputs == NULL)
     return ctf_set_errno (fp, ENOMEM);
 
+  /* Create empty CUs if requested.  We do not currently claim that multiple
+     links in succession with CTF_LINK_EMPTY_CU_MAPPINGS set in some calls and
+     not set in others will do anything especially sensible.  */
+
+  if (fp->ctf_link_out_cu_mapping && (flags & CTF_LINK_EMPTY_CU_MAPPINGS))
+    {
+      void *v;
+
+      while ((err = ctf_dynhash_next (fp->ctf_link_out_cu_mapping, &i, &v,
+				      NULL)) == 0)
+	{
+	  const char *to = (const char *) v;
+	  if (ctf_create_per_cu (fp, to, to) == NULL)
+	    {
+	      ctf_next_destroy (i);
+	      return -1;			/* Errno is set for us.  */
+	    }
+	}
+      if (err != ECTF_NEXT_END)
+	{
+	  ctf_err_warn (fp, 1, "Iteration error creating empty CUs: %s",
+			ctf_errmsg (err));
+	  ctf_set_errno (fp, err);
+	  return -1;
+	}
+    }
+
   ctf_dynhash_iter (fp->ctf_link_inputs, ctf_link_one_input_archive,
 		    &arg);
 
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index 87bff2f122d..285e0e0ff71 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1716,7 +1716,8 @@ ctf_file_close (ctf_file_t *fp)
   ctf_dynhash_destroy (fp->ctf_link_inputs);
   ctf_dynhash_destroy (fp->ctf_link_outputs);
   ctf_dynhash_destroy (fp->ctf_link_type_mapping);
-  ctf_dynhash_destroy (fp->ctf_link_cu_mapping);
+  ctf_dynhash_destroy (fp->ctf_link_in_cu_mapping);
+  ctf_dynhash_destroy (fp->ctf_link_out_cu_mapping);
   ctf_dynhash_destroy (fp->ctf_add_processing);
 
   for (err = ctf_list_next (&fp->ctf_errs_warnings); err != NULL; err = nerr)
-- 
2.27.0.247.g3dff7de930

[PATCH 46/59] ctf, link: fix spurious conflicts of variables in the variable section

[Nick Alcock]

When we link a CTF variable, we check to see if it already exists in the
parent dict first: if it does, and it has a type the same as the type we
would populate it with, we assume we don't need to do anything:
otherwise, we populate it in a per-CU child.

Or that's what we should be doing.  Instead, we check if the type is the
same as the type in *source dict*, which is going to be a completely
different value!  So we end up concluding all variables are conflicting,
bloating up output possibly quite a lot (variables aren't big in and of
themselves, but each drags around a strtab entry, and CTF dicts in a CTF
archive do not share their strtabs -- one of many problems with CTF
archives as presently constituted.)

Fix trivial: check the right type.

libctf/
	* ctf-link.c (ctf_link_one_variable): Check the dst_type for
	conflicts, not the source type.
---
 libctf/ctf-link.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 3c96604b36a..7b0ac386dec 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -584,7 +584,7 @@ ctf_link_one_variable (const char *name, ctf_id_t type, void *arg_)
 	    }
 
 	  /* Already present?  Nothing to do.  */
-	  if (dvd && dvd->dvd_type == type)
+	  if (dvd && dvd->dvd_type == dst_type)
 	    return 0;
 	}
     }
-- 
2.27.0.247.g3dff7de930

[PATCH 47/59] libctf, link: add the ability to filter out variables from the link

[Nick Alcock]

The CTF variables section (containing variables that have no
corresponding symtab entries) can cause the string table to get very
voluminous if the names of variables are long.  Some callers want to
filter out particular variables they know they won't need.

So add a "variable filter" callback that does that: it's passed the name
of the variable and a corresponding ctf_file_t / ctf_id_t pair, and
should return 1 to filter it out.

ld doesn't use this machinery yet, but we could easily add it later if
desired.  (But see later for a commit that turns off CTF variable-
section linking in ld entirely by default.)

include/
	* ctf-api.h (ctf_link_variable_filter_t): New.
	(ctf_link_set_variable_filter): Likewise.

libctf/
	* libctf.ver (ctf_link_set_variable_filter): Add.
	* ctf-impl.h (ctf_file_t) <ctf_link_variable_filter>: New.
	<ctf_link_variable_filter_arg>: Likewise.
	* ctf-create.c (ctf_serialize): Adjust.
	* ctf-link.c (ctf_link_set_variable_filter): New, set it.
	(ctf_link_one_variable): Call it if set.
---
 include/ctf-api.h   |  6 ++++++
 libctf/ctf-create.c |  2 ++
 libctf/ctf-impl.h   |  7 ++++++-
 libctf/ctf-link.c   | 19 +++++++++++++++++++
 libctf/libctf.ver   |  1 +
 5 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 700a2b1ef5d..70bf7e0d8d2 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -458,6 +458,12 @@ extern int ctf_compress_write (ctf_file_t * fp, int fd);
 extern unsigned char *ctf_write_mem (ctf_file_t *, size_t *, size_t threshold);
 
 extern int ctf_link_add_ctf (ctf_file_t *, ctf_archive_t *, const char *);
+/* The variable filter should return nonzero if a variable should not
+   appear in the output.  */
+typedef int ctf_link_variable_filter_f (ctf_file_t *, const char *, ctf_id_t,
+					void *);
+extern int ctf_link_set_variable_filter (ctf_file_t *,
+					 ctf_link_variable_filter_f *, void *);
 extern int ctf_link (ctf_file_t *, int flags);
 typedef const char *ctf_link_strtab_string_f (uint32_t *offset, void *arg);
 extern int ctf_link_add_strtab (ctf_file_t *, ctf_link_strtab_string_f *,
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 10c6bbf552e..001e625aa87 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -543,6 +543,8 @@ ctf_serialize (ctf_file_t *fp)
   nfp->ctf_link_type_mapping = fp->ctf_link_type_mapping;
   nfp->ctf_link_memb_name_changer = fp->ctf_link_memb_name_changer;
   nfp->ctf_link_memb_name_changer_arg = fp->ctf_link_memb_name_changer_arg;
+  nfp->ctf_link_variable_filter = fp->ctf_link_variable_filter;
+  nfp->ctf_link_variable_filter_arg = fp->ctf_link_variable_filter_arg;
   nfp->ctf_link_flags = fp->ctf_link_flags;
 
   nfp->ctf_snapshot_lu = fp->ctf_snapshots;
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 46bceb49861..7f47c68eccb 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -333,7 +333,12 @@ struct ctf_file
 
   /* Allow the caller to change the name of link archive members.  */
   ctf_link_memb_name_changer_f *ctf_link_memb_name_changer;
-  void *ctf_link_memb_name_changer_arg; /* Argument for it.  */
+  void *ctf_link_memb_name_changer_arg;         /* Argument for it.  */
+
+  /* Allow the caller to filter out variables they don't care about.  */
+  ctf_link_variable_filter_f *ctf_link_variable_filter;
+  void *ctf_link_variable_filter_arg;           /* Argument for it. */
+
   ctf_dynhash_t *ctf_add_processing; /* Types ctf_add_type is working on now.  */
   char *ctf_tmp_typeslice;	  /* Storage for slicing up type names.  */
   size_t ctf_tmp_typeslicelen;	  /* Size of the typeslice.  */
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 7b0ac386dec..886106cb478 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -528,6 +528,16 @@ ctf_link_one_type (ctf_id_t type, int isroot _libctf_unused_, void *arg_)
   return 0;					/* As above: do not lose types.  */
 }
 
+/* Set a function which is used to filter out unwanted variables from the link.  */
+int
+ctf_link_set_variable_filter (ctf_file_t *fp, ctf_link_variable_filter_f *filter,
+			      void *arg)
+{
+  fp->ctf_link_variable_filter = filter;
+  fp->ctf_link_variable_filter_arg = arg;
+  return 0;
+}
+
 /* Check if we can safely add a variable with the given type to this container.  */
 
 static int
@@ -564,6 +574,15 @@ ctf_link_one_variable (const char *name, ctf_id_t type, void *arg_)
   ctf_file_t *check_fp;
   ctf_dvdef_t *dvd;
 
+  /* See if this variable is filtered out.  */
+
+  if (arg->out_fp->ctf_link_variable_filter)
+    {
+      void *farg = arg->out_fp->ctf_link_variable_filter_arg;
+      if (arg->out_fp->ctf_link_variable_filter (arg->in_fp, name, type, farg))
+	return 0;
+    }
+
   /* In unconflicted link mode, if this type is mapped to a type in the parent
      container, we want to try to add to that first: if it reports a duplicate,
      or if the type is in a child already, add straight to the child.  */
diff --git a/libctf/libctf.ver b/libctf/libctf.ver
index 7eed14ae54d..62f99771d9b 100644
--- a/libctf/libctf.ver
+++ b/libctf/libctf.ver
@@ -159,6 +159,7 @@ LIBCTF_1.0 {
 	ctf_link_add_ctf;
 	ctf_link_add_cu_mapping;
 	ctf_link_set_memb_name_changer;
+	ctf_link_set_variable_filter;
 	ctf_link;
 	ctf_link_add_strtab;
 	ctf_link_shuffle_syms;
-- 
2.27.0.247.g3dff7de930

[PATCH 48/59] libctf: add SHA-1 support for libctf

[Nick Alcock]

This very thin abstraction layer provides SHA-1ing facilities to all of
libctf, almost all inlined wrappers around the libiberty functionality
other than ctf_sha1_fini.

The deduplicator will use this to recursively hash types to prove their
identity.

libctf/
	* ctf-sha1.h: New, inline wrappers around sha1_init_ctx and
	sha1_process_bytes.
	* ctf-impl.h: Include it.
	(ctf_sha1_init): New.
	(ctf_sha1_add): Likewise.
	(ctf_sha1_fini): Likewise.
	* ctf-sha1.c: New, non-inline wrapper around sha1_finish_ctx
	producing strings.
	* Makefile.am: Add file.
	* Makefile.in: Regenerate.
---
 libctf/Makefile.am |  2 +-
 libctf/Makefile.in | 40 +++++++++++++++++++++++++++----------
 libctf/ctf-impl.h  |  5 +++++
 libctf/ctf-sha1.c  | 50 ++++++++++++++++++++++++++++++++++++++++++++++
 libctf/ctf-sha1.h  | 41 +++++++++++++++++++++++++++++++++++++
 5 files changed, 126 insertions(+), 12 deletions(-)
 create mode 100644 libctf/ctf-sha1.c
 create mode 100644 libctf/ctf-sha1.h

diff --git a/libctf/Makefile.am b/libctf/Makefile.am
index de27cf82394..8f5ea6b0247 100644
--- a/libctf/Makefile.am
+++ b/libctf/Makefile.am
@@ -44,7 +44,7 @@ libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
 libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c ctf-decl.c ctf-error.c \
 			  ctf-hash.c ctf-labels.c ctf-link.c ctf-lookup.c ctf-open.c \
-			  ctf-string.c ctf-subr.c ctf-types.c ctf-util.c
+			  ctf-sha1.c ctf-string.c ctf-subr.c ctf-types.c ctf-util.c
 if NEED_CTF_QSORT_R
 libctf_nobfd_la_SOURCES += ctf-qsort_r.c
 endif
diff --git a/libctf/Makefile.in b/libctf/Makefile.in
index 3b7feea02d2..328edcd5511 100644
--- a/libctf/Makefile.in
+++ b/libctf/Makefile.in
@@ -165,17 +165,18 @@ am__DEPENDENCIES_1 =
 libctf_nobfd_la_DEPENDENCIES = $(am__DEPENDENCIES_1)
 am__libctf_nobfd_la_SOURCES_DIST = ctf-archive.c ctf-dump.c \
 	ctf-create.c ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c \
-	ctf-link.c ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c \
-	ctf-types.c ctf-util.c ctf-qsort_r.c
+	ctf-link.c ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c \
+	ctf-subr.c ctf-types.c ctf-util.c ctf-qsort_r.c
 @NEED_CTF_QSORT_R_TRUE@am__objects_1 = libctf_nobfd_la-ctf-qsort_r.lo
 am_libctf_nobfd_la_OBJECTS = libctf_nobfd_la-ctf-archive.lo \
 	libctf_nobfd_la-ctf-dump.lo libctf_nobfd_la-ctf-create.lo \
 	libctf_nobfd_la-ctf-decl.lo libctf_nobfd_la-ctf-error.lo \
 	libctf_nobfd_la-ctf-hash.lo libctf_nobfd_la-ctf-labels.lo \
 	libctf_nobfd_la-ctf-link.lo libctf_nobfd_la-ctf-lookup.lo \
-	libctf_nobfd_la-ctf-open.lo libctf_nobfd_la-ctf-string.lo \
-	libctf_nobfd_la-ctf-subr.lo libctf_nobfd_la-ctf-types.lo \
-	libctf_nobfd_la-ctf-util.lo $(am__objects_1)
+	libctf_nobfd_la-ctf-open.lo libctf_nobfd_la-ctf-sha1.lo \
+	libctf_nobfd_la-ctf-string.lo libctf_nobfd_la-ctf-subr.lo \
+	libctf_nobfd_la-ctf-types.lo libctf_nobfd_la-ctf-util.lo \
+	$(am__objects_1)
 libctf_nobfd_la_OBJECTS = $(am_libctf_nobfd_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
 am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@)
@@ -190,16 +191,17 @@ libctf_nobfd_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \
 am__DEPENDENCIES_2 = $(am__DEPENDENCIES_1)
 am__libctf_la_SOURCES_DIST = ctf-archive.c ctf-dump.c ctf-create.c \
 	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
-	ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c ctf-types.c \
-	ctf-util.c ctf-qsort_r.c ctf-open-bfd.c
+	ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c ctf-subr.c \
+	ctf-types.c ctf-util.c ctf-qsort_r.c ctf-open-bfd.c
 @NEED_CTF_QSORT_R_TRUE@am__objects_2 = libctf_la-ctf-qsort_r.lo
 am__objects_3 = libctf_la-ctf-archive.lo libctf_la-ctf-dump.lo \
 	libctf_la-ctf-create.lo libctf_la-ctf-decl.lo \
 	libctf_la-ctf-error.lo libctf_la-ctf-hash.lo \
 	libctf_la-ctf-labels.lo libctf_la-ctf-link.lo \
 	libctf_la-ctf-lookup.lo libctf_la-ctf-open.lo \
-	libctf_la-ctf-string.lo libctf_la-ctf-subr.lo \
-	libctf_la-ctf-types.lo libctf_la-ctf-util.lo $(am__objects_2)
+	libctf_la-ctf-sha1.lo libctf_la-ctf-string.lo \
+	libctf_la-ctf-subr.lo libctf_la-ctf-types.lo \
+	libctf_la-ctf-util.lo $(am__objects_2)
 am_libctf_la_OBJECTS = $(am__objects_3) libctf_la-ctf-open-bfd.lo
 libctf_la_OBJECTS = $(am_libctf_la_OBJECTS)
 libctf_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
@@ -455,8 +457,8 @@ libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
 libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c \
 	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
-	ctf-lookup.c ctf-open.c ctf-string.c ctf-subr.c ctf-types.c \
-	ctf-util.c $(am__append_1)
+	ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c ctf-subr.c \
+	ctf-types.c ctf-util.c $(am__append_1)
 libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
 libctf_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=0
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
@@ -587,6 +589,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-open-bfd.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-open.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-qsort_r.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-sha1.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-string.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-subr.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-types.Plo@am__quote@
@@ -602,6 +605,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-lookup.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-open.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-qsort_r.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-sha1.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-string.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-subr.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-types.Plo@am__quote@
@@ -698,6 +702,13 @@ libctf_nobfd_la-ctf-open.lo: ctf-open.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
 
+libctf_nobfd_la-ctf-sha1.lo: ctf-sha1.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-sha1.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-sha1.Tpo -c -o libctf_nobfd_la-ctf-sha1.lo `test -f 'ctf-sha1.c' || echo '$(srcdir)/'`ctf-sha1.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-sha1.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-sha1.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-sha1.c' object='libctf_nobfd_la-ctf-sha1.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-sha1.lo `test -f 'ctf-sha1.c' || echo '$(srcdir)/'`ctf-sha1.c
+
 libctf_nobfd_la-ctf-string.lo: ctf-string.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-string.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-string.Tpo -c -o libctf_nobfd_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-string.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-string.Plo
@@ -803,6 +814,13 @@ libctf_la-ctf-open.lo: ctf-open.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-open.lo `test -f 'ctf-open.c' || echo '$(srcdir)/'`ctf-open.c
 
+libctf_la-ctf-sha1.lo: ctf-sha1.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-sha1.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-sha1.Tpo -c -o libctf_la-ctf-sha1.lo `test -f 'ctf-sha1.c' || echo '$(srcdir)/'`ctf-sha1.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-sha1.Tpo $(DEPDIR)/libctf_la-ctf-sha1.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-sha1.c' object='libctf_la-ctf-sha1.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-sha1.lo `test -f 'ctf-sha1.c' || echo '$(srcdir)/'`ctf-sha1.c
+
 libctf_la-ctf-string.lo: ctf-string.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-string.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-string.Tpo -c -o libctf_la-ctf-string.lo `test -f 'ctf-string.c' || echo '$(srcdir)/'`ctf-string.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-string.Tpo $(DEPDIR)/libctf_la-ctf-string.Plo
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 7f47c68eccb..9c5f7bac65a 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -25,6 +25,7 @@
 #include <sys/param.h>
 #include "ctf-decls.h"
 #include <ctf-api.h>
+#include "ctf-sha1.h"
 #include <sys/types.h>
 #include <stdlib.h>
 #include <stdarg.h>
@@ -499,6 +500,10 @@ extern int ctf_dynset_exists (ctf_dynset_t *, const void *key,
 extern int ctf_dynset_next (ctf_dynset_t *, ctf_next_t **, void **key);
 extern void *ctf_dynset_lookup_any (ctf_dynset_t *);
 
+extern void ctf_sha1_init (ctf_sha1_t *);
+extern void ctf_sha1_add (ctf_sha1_t *, const void *, size_t);
+extern char *ctf_sha1_fini (ctf_sha1_t *, char *);
+
 #define	ctf_list_prev(elem)	((void *)(((ctf_list_t *)(elem))->l_prev))
 #define	ctf_list_next(elem)	((void *)(((ctf_list_t *)(elem))->l_next))
 
diff --git a/libctf/ctf-sha1.c b/libctf/ctf-sha1.c
new file mode 100644
index 00000000000..47e66558a46
--- /dev/null
+++ b/libctf/ctf-sha1.c
@@ -0,0 +1,50 @@
+/* SHA-1 thunks.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+
+   This file is part of libctf.
+
+   libctf is free software; you can redistribute it and/or modify it under
+   the terms of the GNU General Public License as published by the Free
+   Software Foundation; either version 3, or (at your option) any later
+   version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+   See the GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; see the file COPYING.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <ctf-impl.h>
+#include <ctf-sha1.h>
+
+static const char hex[] = "0123456789abcdef";
+
+char *
+ctf_sha1_fini (ctf_sha1_t *sha1, char *buf)
+{
+  size_t i;
+
+  /* Alignment suitable for a uint32_t. */
+  union
+  {
+    uint32_t align;
+    unsigned char digest[((CTF_SHA1_SIZE - 1) / 2) + 1];
+  } align;
+
+  sha1_finish_ctx (sha1, align.digest);
+
+  if (!buf)
+    return NULL;
+
+  buf[CTF_SHA1_SIZE - 1] = '\0';
+
+  for (i = 0; i < (CTF_SHA1_SIZE - 1) / 2; i++)
+    {
+      buf[2 * i] = hex[align.digest[i] >> 4];
+      buf[2 * i + 1] = hex[align.digest[i] & 0xf];
+    }
+  return buf;
+}
diff --git a/libctf/ctf-sha1.h b/libctf/ctf-sha1.h
new file mode 100644
index 00000000000..b9e0c9eedf8
--- /dev/null
+++ b/libctf/ctf-sha1.h
@@ -0,0 +1,41 @@
+/* SHA-1 thunks.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+
+   This file is part of libctf.
+
+   libctf is free software; you can redistribute it and/or modify it under
+   the terms of the GNU General Public License as published by the Free
+   Software Foundation; either version 3, or (at your option) any later
+   version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+   See the GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; see the file COPYING.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#ifndef _CTF_SHA1_H
+#define _CTF_SHA1_H
+
+#include "config.h"
+#include "sha1.h"
+
+#define CTF_SHA1_SIZE 41
+
+typedef struct sha1_ctx ctf_sha1_t;
+
+static inline void
+ctf_sha1_init (ctf_sha1_t *sha1)
+{
+  sha1_init_ctx (sha1);
+}
+
+static inline void
+ctf_sha1_add (ctf_sha1_t *sha1, const void *buf, size_t len)
+{
+  sha1_process_bytes (buf, len, sha1);
+}
+#endif
-- 
2.27.0.247.g3dff7de930

[PATCH 49/59] libctf, dedup: add new configure option --enable-libctf-hash-debugging

[Nick Alcock]

Add a new debugging configure option, --enable-libctf-hash-debugging,
off by default, which lets you configure in expensive internal
consistency checks and enable the printing of debugging output when
LIBCTF_DEBUG=t before type deduplication has happened.

In this commit we just add the option and cause it to turn ctf_assert
into a real, hard assert for easier debugging.

libctf/
	* configure.ac: Add --enable-libctf-hash-debugging.
	* aclocal.m4: Pull in enable.m4, for GCC_ENABLE.
	* Makefile.in: Regenerated.
	* configure: Likewise.
	* config.h.in: Likewise.
	* ctf-impl.h [ENABLE_LIBCTF_HASH_DEBUGGING]
	(ctf_assert): Define to assert.
---
 libctf/Makefile.in  |  1 +
 libctf/aclocal.m4   |  1 +
 libctf/config.h.in  |  3 +++
 libctf/configure    | 41 +++++++++++++++++++++++++++++++++++++++--
 libctf/configure.ac |  6 ++++++
 libctf/ctf-impl.h   |  5 +++++
 6 files changed, 55 insertions(+), 2 deletions(-)

diff --git a/libctf/Makefile.in b/libctf/Makefile.in
index 328edcd5511..bc385278e09 100644
--- a/libctf/Makefile.in
+++ b/libctf/Makefile.in
@@ -113,6 +113,7 @@ ACLOCAL_M4 = $(top_srcdir)/aclocal.m4
 am__aclocal_m4_deps = $(top_srcdir)/../bfd/acinclude.m4 \
 	$(top_srcdir)/../config/acx.m4 \
 	$(top_srcdir)/../config/depstand.m4 \
+	$(top_srcdir)/../config/enable.m4 \
 	$(top_srcdir)/../config/gettext-sister.m4 \
 	$(top_srcdir)/../config/lead-dot.m4 \
 	$(top_srcdir)/../config/override.m4 \
diff --git a/libctf/aclocal.m4 b/libctf/aclocal.m4
index 8ae4b534629..df51584d837 100644
--- a/libctf/aclocal.m4
+++ b/libctf/aclocal.m4
@@ -1230,6 +1230,7 @@ AC_SUBST([am__untar])
 m4_include([../bfd/acinclude.m4])
 m4_include([../config/acx.m4])
 m4_include([../config/depstand.m4])
+m4_include([../config/enable.m4])
 m4_include([../config/gettext-sister.m4])
 m4_include([../config/lead-dot.m4])
 m4_include([../config/override.m4])
diff --git a/libctf/config.h.in b/libctf/config.h.in
index 897587e5875..2ddd5126088 100644
--- a/libctf/config.h.in
+++ b/libctf/config.h.in
@@ -3,6 +3,9 @@
 /* Define if building universal (internal helper macro) */
 #undef AC_APPLE_UNIVERSAL_BUILD
 
+/* Enable expensive debugging of CTF deduplication type hashing */
+#undef ENABLE_LIBCTF_HASH_DEBUGGING
+
 /* Define to 1 if translation of program messages to the user's native
    language is requested. */
 #undef ENABLE_NLS
diff --git a/libctf/configure b/libctf/configure
index 58aaa3a529b..9f16ae92b8d 100755
--- a/libctf/configure
+++ b/libctf/configure
@@ -642,6 +642,8 @@ SHARED_LIBADD
 SHARED_LDFLAGS
 NEED_CTF_QSORT_R_FALSE
 NEED_CTF_QSORT_R_TRUE
+ENABLE_LIBCTF_HASH_DEBUGGING_FALSE
+ENABLE_LIBCTF_HASH_DEBUGGING_TRUE
 zlibinc
 zlibdir
 ac_libctf_warn_cflags
@@ -796,6 +798,7 @@ enable_werror_always
 enable_maintainer_mode
 enable_install_libbfd
 with_system_zlib
+enable_libctf_hash_debugging
 '
       ac_precious_vars='build_alias
 host_alias
@@ -1441,6 +1444,9 @@ Optional Features:
                           enable make rules and dependencies not useful (and
                           sometimes confusing) to the casual installer
   --enable-install-libbfd controls installation of libbfd and related headers
+  --enable-libctf-hash-debugging
+                          Enable expensive debugging of CTF deduplication type
+                          hashing [default=no]
 
 Optional Packages:
   --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
@@ -11513,7 +11519,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11516 "configure"
+#line 11522 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -11619,7 +11625,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 11622 "configure"
+#line 11628 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -12799,6 +12805,33 @@ fi
 
 
 
+ # Check whether --enable-libctf-hash-debugging was given.
+if test "${enable_libctf_hash_debugging+set}" = set; then :
+  enableval=$enable_libctf_hash_debugging;
+      case "$enableval" in
+       yes|no) ;;
+       *) as_fn_error $? "Argument to enable/disable libctf-hash-debugging must be yes or no" "$LINENO" 5 ;;
+      esac
+
+else
+  enable_libctf_hash_debugging=no
+fi
+
+
+if test "${enable_libctf_hash_debugging}" = yes; then
+
+$as_echo "#define ENABLE_LIBCTF_HASH_DEBUGGING 1" >>confdefs.h
+
+fi
+ if test "${enable_libctf_hash_debugging}" = yes; then
+  ENABLE_LIBCTF_HASH_DEBUGGING_TRUE=
+  ENABLE_LIBCTF_HASH_DEBUGGING_FALSE='#'
+else
+  ENABLE_LIBCTF_HASH_DEBUGGING_TRUE='#'
+  ENABLE_LIBCTF_HASH_DEBUGGING_FALSE=
+fi
+
+
 # Similar to GDB_AC_CHECK_BFD.
 OLD_CFLAGS=$CFLAGS
 OLD_LDFLAGS=$LDFLAGS
@@ -13495,6 +13528,10 @@ if test -z "${INSTALL_LIBBFD_TRUE}" && test -z "${INSTALL_LIBBFD_FALSE}"; then
   as_fn_error $? "conditional \"INSTALL_LIBBFD\" was never defined.
 Usually this means the macro was only invoked conditionally." "$LINENO" 5
 fi
+if test -z "${ENABLE_LIBCTF_HASH_DEBUGGING_TRUE}" && test -z "${ENABLE_LIBCTF_HASH_DEBUGGING_FALSE}"; then
+  as_fn_error $? "conditional \"ENABLE_LIBCTF_HASH_DEBUGGING\" was never defined.
+Usually this means the macro was only invoked conditionally." "$LINENO" 5
+fi
 
 if test -z "${NEED_CTF_QSORT_R_TRUE}" && test -z "${NEED_CTF_QSORT_R_FALSE}"; then
   as_fn_error $? "conditional \"NEED_CTF_QSORT_R\" was never defined.
diff --git a/libctf/configure.ac b/libctf/configure.ac
index 26b062e7a54..3799a0cb906 100644
--- a/libctf/configure.ac
+++ b/libctf/configure.ac
@@ -67,6 +67,12 @@ AC_FUNC_MMAP
 AC_SEARCH_LIBS(dlopen, dl)
 AM_ZLIB
 
+GCC_ENABLE([libctf-hash-debugging], [no], [], [Enable expensive debugging of CTF deduplication type hashing])
+if test "${enable_libctf_hash_debugging}" = yes; then
+    AC_DEFINE(ENABLE_LIBCTF_HASH_DEBUGGING, 1, [Enable expensive debugging of CTF deduplication type hashing])
+fi
+AM_CONDITIONAL(ENABLE_LIBCTF_HASH_DEBUGGING, test "${enable_libctf_hash_debugging}" = yes)
+
 # Similar to GDB_AC_CHECK_BFD.
 OLD_CFLAGS=$CFLAGS
 OLD_LDFLAGS=$LDFLAGS
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 9c5f7bac65a..4dea3f018f0 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -71,9 +71,14 @@ extern "C"
 
 #endif
 
+#if defined ENABLE_LIBCTF_HASH_DEBUGGING && !defined (NDEBUG)
+#include <assert.h>
+#define ctf_assert(fp, expr) (assert (expr), 1)
+#else
 #define ctf_assert(fp, expr)						\
   _libctf_unlikely_ (ctf_assert_internal (fp, __FILE__, __LINE__,	\
 					  #expr, !!(expr)))
+#endif
 
 /* libctf in-memory state.  */
 
-- 
2.27.0.247.g3dff7de930

[PATCH 50/59] libctf, dedup: add deduplicator

[Nick Alcock]

This adds the core deduplicator that the ctf_link machinery calls
(possibly repeatedly) to link the CTF sections: it takes an array
of input ctf_file_t's and another array that indicates which entries in
the input array are parents of which other entries, and returns an array
of outputs.  The first output is always the ctf_file_t on which
ctf_link/ctf_dedup/etc was called: the other outputs are child dicts
that have the first output as their parent.

include/
	* ctf-api.h (CTF_LINK_SHARE_DUPLICATED): No longer unimplemented.
libctf/
	* ctf-impl.h (ctf_type_id_key): New, the key in the
	cd_id_to_file_t.
	(ctf_dedup): New, core deduplicator state.
	(ctf_file_t) <ctf_dedup>: New.
	<ctf_dedup_atoms>: New.
	<ctf_dedup_atoms_alloc>: New.
	(ctf_hash_type_id_key): New prototype.
	(ctf_hash_eq_type_id_key): Likewise.
	(ctf_dedup_atoms_init): Likewise.
	* ctf-hash.c (ctf_hash_eq_type_id_key): New.
	(ctf_dedup_atoms_init): Likewise.
	* ctf-create.c (ctf_serialize): Adjusted.
	(ctf_add_encoded): No longer static.
	(ctf_add_reftype): Likewise.
	* ctf-open.c (ctf_file_close): Destroy the
	ctf_dedup_atoms_alloc.
	* ctf-dedup.c: New file.
	* Makefile.am: Add it.
	* Makefile.in: Regenerate.
---
 include/ctf-api.h   |    2 +-
 libctf/Makefile.am  |    5 +-
 libctf/Makefile.in  |   52 +-
 libctf/ctf-create.c |   10 +-
 libctf/ctf-dedup.c  | 3157 +++++++++++++++++++++++++++++++++++++++++++
 libctf/ctf-hash.c   |   22 +
 libctf/ctf-impl.h   |  127 ++
 libctf/ctf-open.c   |    2 +
 8 files changed, 3354 insertions(+), 23 deletions(-)
 create mode 100644 libctf/ctf-dedup.c

diff --git a/include/ctf-api.h b/include/ctf-api.h
index 70bf7e0d8d2..c3683db0996 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -83,7 +83,7 @@ typedef struct ctf_link_sym
 /* Share all types that are not in conflict.  The default.  */
 #define CTF_LINK_SHARE_UNCONFLICTED 0x0
 
-/* Share only types that are used by multiple inputs.  Not implemented yet.  */
+/* Share only types that are used by multiple inputs.  */
 #define CTF_LINK_SHARE_DUPLICATED 0x1
 
 /* Create empty outputs for all registered CU mappings even if no types are
diff --git a/libctf/Makefile.am b/libctf/Makefile.am
index 8f5ea6b0247..8b8f0a8cf83 100644
--- a/libctf/Makefile.am
+++ b/libctf/Makefile.am
@@ -43,8 +43,9 @@ libctf_nobfd_la_LIBADD = @SHARED_LIBADD@ $(ZLIB)
 libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
 libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c ctf-decl.c ctf-error.c \
-			  ctf-hash.c ctf-labels.c ctf-link.c ctf-lookup.c ctf-open.c \
-			  ctf-sha1.c ctf-string.c ctf-subr.c ctf-types.c ctf-util.c
+			  ctf-hash.c ctf-labels.c ctf-dedup.c ctf-link.c ctf-lookup.c \
+			  ctf-open.c ctf-sha1.c ctf-string.c ctf-subr.c ctf-types.c \
+			  ctf-util.c
 if NEED_CTF_QSORT_R
 libctf_nobfd_la_SOURCES += ctf-qsort_r.c
 endif
diff --git a/libctf/Makefile.in b/libctf/Makefile.in
index bc385278e09..78de09ba10a 100644
--- a/libctf/Makefile.in
+++ b/libctf/Makefile.in
@@ -166,18 +166,18 @@ am__DEPENDENCIES_1 =
 libctf_nobfd_la_DEPENDENCIES = $(am__DEPENDENCIES_1)
 am__libctf_nobfd_la_SOURCES_DIST = ctf-archive.c ctf-dump.c \
 	ctf-create.c ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c \
-	ctf-link.c ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c \
-	ctf-subr.c ctf-types.c ctf-util.c ctf-qsort_r.c
+	ctf-dedup.c ctf-link.c ctf-lookup.c ctf-open.c ctf-sha1.c \
+	ctf-string.c ctf-subr.c ctf-types.c ctf-util.c ctf-qsort_r.c
 @NEED_CTF_QSORT_R_TRUE@am__objects_1 = libctf_nobfd_la-ctf-qsort_r.lo
 am_libctf_nobfd_la_OBJECTS = libctf_nobfd_la-ctf-archive.lo \
 	libctf_nobfd_la-ctf-dump.lo libctf_nobfd_la-ctf-create.lo \
 	libctf_nobfd_la-ctf-decl.lo libctf_nobfd_la-ctf-error.lo \
 	libctf_nobfd_la-ctf-hash.lo libctf_nobfd_la-ctf-labels.lo \
-	libctf_nobfd_la-ctf-link.lo libctf_nobfd_la-ctf-lookup.lo \
-	libctf_nobfd_la-ctf-open.lo libctf_nobfd_la-ctf-sha1.lo \
-	libctf_nobfd_la-ctf-string.lo libctf_nobfd_la-ctf-subr.lo \
-	libctf_nobfd_la-ctf-types.lo libctf_nobfd_la-ctf-util.lo \
-	$(am__objects_1)
+	libctf_nobfd_la-ctf-dedup.lo libctf_nobfd_la-ctf-link.lo \
+	libctf_nobfd_la-ctf-lookup.lo libctf_nobfd_la-ctf-open.lo \
+	libctf_nobfd_la-ctf-sha1.lo libctf_nobfd_la-ctf-string.lo \
+	libctf_nobfd_la-ctf-subr.lo libctf_nobfd_la-ctf-types.lo \
+	libctf_nobfd_la-ctf-util.lo $(am__objects_1)
 libctf_nobfd_la_OBJECTS = $(am_libctf_nobfd_la_OBJECTS)
 AM_V_lt = $(am__v_lt_@AM_V@)
 am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@)
@@ -191,18 +191,18 @@ libctf_nobfd_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \
 @INSTALL_LIBBFD_TRUE@am_libctf_nobfd_la_rpath = -rpath $(libdir)
 am__DEPENDENCIES_2 = $(am__DEPENDENCIES_1)
 am__libctf_la_SOURCES_DIST = ctf-archive.c ctf-dump.c ctf-create.c \
-	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
-	ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c ctf-subr.c \
-	ctf-types.c ctf-util.c ctf-qsort_r.c ctf-open-bfd.c
+	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-dedup.c \
+	ctf-link.c ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c \
+	ctf-subr.c ctf-types.c ctf-util.c ctf-qsort_r.c ctf-open-bfd.c
 @NEED_CTF_QSORT_R_TRUE@am__objects_2 = libctf_la-ctf-qsort_r.lo
 am__objects_3 = libctf_la-ctf-archive.lo libctf_la-ctf-dump.lo \
 	libctf_la-ctf-create.lo libctf_la-ctf-decl.lo \
 	libctf_la-ctf-error.lo libctf_la-ctf-hash.lo \
-	libctf_la-ctf-labels.lo libctf_la-ctf-link.lo \
-	libctf_la-ctf-lookup.lo libctf_la-ctf-open.lo \
-	libctf_la-ctf-sha1.lo libctf_la-ctf-string.lo \
-	libctf_la-ctf-subr.lo libctf_la-ctf-types.lo \
-	libctf_la-ctf-util.lo $(am__objects_2)
+	libctf_la-ctf-labels.lo libctf_la-ctf-dedup.lo \
+	libctf_la-ctf-link.lo libctf_la-ctf-lookup.lo \
+	libctf_la-ctf-open.lo libctf_la-ctf-sha1.lo \
+	libctf_la-ctf-string.lo libctf_la-ctf-subr.lo \
+	libctf_la-ctf-types.lo libctf_la-ctf-util.lo $(am__objects_2)
 am_libctf_la_OBJECTS = $(am__objects_3) libctf_la-ctf-open-bfd.lo
 libctf_la_OBJECTS = $(am_libctf_la_OBJECTS)
 libctf_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \
@@ -457,9 +457,9 @@ libctf_nobfd_la_LIBADD = @SHARED_LIBADD@ $(ZLIB)
 libctf_nobfd_la_LDFLAGS = -version-info 0:0:0 @SHARED_LDFLAGS@ @VERSION_FLAGS@
 libctf_nobfd_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=1
 libctf_nobfd_la_SOURCES = ctf-archive.c ctf-dump.c ctf-create.c \
-	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-link.c \
-	ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c ctf-subr.c \
-	ctf-types.c ctf-util.c $(am__append_1)
+	ctf-decl.c ctf-error.c ctf-hash.c ctf-labels.c ctf-dedup.c \
+	ctf-link.c ctf-lookup.c ctf-open.c ctf-sha1.c ctf-string.c \
+	ctf-subr.c ctf-types.c ctf-util.c $(am__append_1)
 libctf_la_LIBADD = @BFD_LIBADD@ $(libctf_nobfd_la_LIBADD)
 libctf_la_CPPFLAGS = $(AM_CPPFLAGS) -DNOBFD=0
 libctf_la_DEPENDENCIES = @BFD_DEPENDENCIES@
@@ -581,6 +581,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-archive.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-create.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-decl.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-dedup.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-dump.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-error.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_la-ctf-hash.Plo@am__quote@
@@ -598,6 +599,7 @@ distclean-compile:
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-archive.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-create.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-decl.Plo@am__quote@
+@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-dedup.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-dump.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-error.Plo@am__quote@
 @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libctf_nobfd_la-ctf-hash.Plo@am__quote@
@@ -682,6 +684,13 @@ libctf_nobfd_la-ctf-labels.lo: ctf-labels.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
 
+libctf_nobfd_la-ctf-dedup.lo: ctf-dedup.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-dedup.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-dedup.Tpo -c -o libctf_nobfd_la-ctf-dedup.lo `test -f 'ctf-dedup.c' || echo '$(srcdir)/'`ctf-dedup.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-dedup.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-dedup.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-dedup.c' object='libctf_nobfd_la-ctf-dedup.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_nobfd_la-ctf-dedup.lo `test -f 'ctf-dedup.c' || echo '$(srcdir)/'`ctf-dedup.c
+
 libctf_nobfd_la-ctf-link.lo: ctf-link.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_nobfd_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_nobfd_la-ctf-link.lo -MD -MP -MF $(DEPDIR)/libctf_nobfd_la-ctf-link.Tpo -c -o libctf_nobfd_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_nobfd_la-ctf-link.Tpo $(DEPDIR)/libctf_nobfd_la-ctf-link.Plo
@@ -794,6 +803,13 @@ libctf_la-ctf-labels.lo: ctf-labels.c
 @AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
 @am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-labels.lo `test -f 'ctf-labels.c' || echo '$(srcdir)/'`ctf-labels.c
 
+libctf_la-ctf-dedup.lo: ctf-dedup.c
+@am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-dedup.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-dedup.Tpo -c -o libctf_la-ctf-dedup.lo `test -f 'ctf-dedup.c' || echo '$(srcdir)/'`ctf-dedup.c
+@am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-dedup.Tpo $(DEPDIR)/libctf_la-ctf-dedup.Plo
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	$(AM_V_CC)source='ctf-dedup.c' object='libctf_la-ctf-dedup.lo' libtool=yes @AMDEPBACKSLASH@
+@AMDEP_TRUE@@am__fastdepCC_FALSE@	DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@
+@am__fastdepCC_FALSE@	$(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o libctf_la-ctf-dedup.lo `test -f 'ctf-dedup.c' || echo '$(srcdir)/'`ctf-dedup.c
+
 libctf_la-ctf-link.lo: ctf-link.c
 @am__fastdepCC_TRUE@	$(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(libctf_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT libctf_la-ctf-link.lo -MD -MP -MF $(DEPDIR)/libctf_la-ctf-link.Tpo -c -o libctf_la-ctf-link.lo `test -f 'ctf-link.c' || echo '$(srcdir)/'`ctf-link.c
 @am__fastdepCC_TRUE@	$(AM_V_at)$(am__mv) $(DEPDIR)/libctf_la-ctf-link.Tpo $(DEPDIR)/libctf_la-ctf-link.Plo
diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 001e625aa87..35c286c3b40 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -546,6 +546,9 @@ ctf_serialize (ctf_file_t *fp)
   nfp->ctf_link_variable_filter = fp->ctf_link_variable_filter;
   nfp->ctf_link_variable_filter_arg = fp->ctf_link_variable_filter_arg;
   nfp->ctf_link_flags = fp->ctf_link_flags;
+  nfp->ctf_dedup_atoms = fp->ctf_dedup_atoms;
+  nfp->ctf_dedup_atoms_alloc = fp->ctf_dedup_atoms_alloc;
+  memcpy (&nfp->ctf_dedup, &fp->ctf_dedup, sizeof (fp->ctf_dedup));
 
   nfp->ctf_snapshot_lu = fp->ctf_snapshots;
 
@@ -571,11 +574,14 @@ ctf_serialize (ctf_file_t *fp)
   fp->ctf_link_in_cu_mapping = NULL;
   fp->ctf_link_out_cu_mapping = NULL;
   fp->ctf_link_type_mapping = NULL;
+  fp->ctf_dedup_atoms = NULL;
+  fp->ctf_dedup_atoms_alloc = NULL;
   fp->ctf_parent_unreffed = 1;
 
   fp->ctf_dvhash = NULL;
   memset (&fp->ctf_dvdefs, 0, sizeof (ctf_list_t));
   memset (fp->ctf_lookups, 0, sizeof (fp->ctf_lookups));
+  memset (&fp->ctf_dedup, 0, sizeof (fp->ctf_dedup));
   fp->ctf_structs.ctn_writable = NULL;
   fp->ctf_unions.ctn_writable = NULL;
   fp->ctf_enums.ctn_writable = NULL;
@@ -878,7 +884,7 @@ clp2 (size_t x)
   return (x + 1);
 }
 
-static ctf_id_t
+ctf_id_t
 ctf_add_encoded (ctf_file_t *fp, uint32_t flag,
 		 const char *name, const ctf_encoding_t *ep, uint32_t kind)
 {
@@ -899,7 +905,7 @@ ctf_add_encoded (ctf_file_t *fp, uint32_t flag,
   return type;
 }
 
-static ctf_id_t
+ctf_id_t
 ctf_add_reftype (ctf_file_t *fp, uint32_t flag, ctf_id_t ref, uint32_t kind)
 {
   ctf_dtdef_t *dtd;
diff --git a/libctf/ctf-dedup.c b/libctf/ctf-dedup.c
new file mode 100644
index 00000000000..c7badcd17a1
--- /dev/null
+++ b/libctf/ctf-dedup.c
@@ -0,0 +1,3157 @@
+/* CTF type deduplication.
+   Copyright (C) 2019 Free Software Foundation, Inc.
+
+   This file is part of libctf.
+
+   libctf is free software; you can redistribute it and/or modify it under
+   the terms of the GNU General Public License as published by the Free
+   Software Foundation; either version 3, or (at your option) any later
+   version.
+
+   This program is distributed in the hope that it will be useful, but
+   WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+   See the GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; see the file COPYING.  If not see
+   <http://www.gnu.org/licenses/>.  */
+
+#include <ctf-impl.h>
+#include <string.h>
+#include <errno.h>
+#include <assert.h>
+#include "hashtab.h"
+
+/* (In the below, relevant functions are named in square brackets.)  */
+
+/* Type deduplication is a three-phase process:
+
+    [ctf_dedup, ctf_dedup_hash_type, ctf_dedup_rhash_type]
+    1) come up with unambiguous hash values for all types: no two types may have
+       the same hash value, and any given type should have only one hash value
+       (for optimal deduplication).
+
+    [ctf_dedup, ctf_dedup_detect_name_ambiguity,
+     ctf_dedup_conflictify_unshared, ctf_dedup_mark_conflicting_hash]
+    2) mark those distinct types with names that collide (and thus cannot be
+       declared simultaneously in the same translation unit) as conflicting, and
+       recursively mark all types that cite one of those types as conflicting as
+       well.  Possibly mark all types cited in only one TU as conflicting, if
+       the CTF_LINK_SHARE_DUPLICATED link mode is active.
+
+    [ctf_dedup_emit, ctf_dedup_emit_struct_members, ctf_dedup_id_to_target]
+    3) emit all the types, one hash value at a time.  Types not marked
+       conflicting are emitted once, into the shared dictionary: types marked
+       conflicting are emitted once per TU into a dictionary corresponding to
+       each TU in which they appear.  Structs marked conflicting get at the very
+       least a forward emitted into the shared dict so that other dicts can cite
+       it if needed.
+
+   [id_to_packed_id]
+   This all works over an array of inputs (usually in the same order as the
+   inputs on the link line).  We don't use the ctf_link_inputs hash directly
+   because it is convenient to be able to address specific input types as a
+   *global type ID* or 'GID', a pair of an array offset and a ctf_id_t.  Since
+   both are already 32 bits or less or can easily be constrained to that range,
+   we can pack them both into a single 64-bit hash word for easy lookups, which
+   would be much more annoying to do with a ctf_file_t * and a ctf_id_t.  (On
+   32-bit platforms, we must do that anyway, since pointers, and thus hash keys
+   and values, are only 32 bits wide).  We track which inputs are parents of
+   which other inputs so that we can correctly recognize that types we have
+   traversed in children may cite types in parents, and so that we can process
+   the parents first.)
+
+   Note that thanks to ld -r, the deduplicator can be fed its own output, so the
+   inputs may themselves have child dicts.  Since we need to support this usage
+   anyway, we can use it in one other place.  If the caller finds translation
+   units to be too small a unit ambiguous types, links can be 'cu-mapped', where
+   the caller provides a mapping of input TU names to output child dict names.
+   This mapping can fuse many child TUs into one potential child dict, so that
+   ambiguous types in any of those input TUs go into the same child dict.
+   When a many:1 cu-mapping is detected, the ctf_dedup machinery is called
+   repeatedly, once for every output name that has more than one input, to fuse
+   all the input TUs associated with a given output dict into one, and once again
+   as normal to deduplicate all those intermediate outputs (and any 1:1 inputs)
+   together.  This has much higher memory usage than otherwise, because in the
+   intermediate state, all the output TUs are in memory at once and cannot be
+   lazily opened.  It also has implications for the emission code: if types
+   appear ambiguously in multiple input TUs that are all mapped to the same
+   child dict, we cannot put them in children in the cu-mapping link phase
+   because this output is meant to *become* a child in the next link stage and
+   parent/child relationships are only one level deep: so instead, we just hide
+   all but one of the ambiguous types.
+
+   There are a few other subtleties here that make this more complex than it
+   seems.  Let's go over the steps above in more detail.
+
+   1) HASHING.
+
+   [ctf_dedup_hash_type, ctf_dedup_rhash_type]
+   Hashing proceeds recursively, mixing in the properties of each input type
+   (including its name, if any), and then adding the hash values of every type
+   cited by that type.  The result is stashed in the cd_type_hashes so other
+   phases can find the hash values of input types given their IDs, and so that
+   if we encounter this type again while hashing we can just return its hash
+   value: it is also stashed in the *output mapping*, a mapping from hash value
+   to the set of GIDs corresponding to that type in all inputs.  We also keep
+   track of the GID of the first appearance of the type in any input (in
+   cd_output_first_gid), and the GID of structs, unions, and forwards that only
+   appear in one TU (in cd_struct_origin).  See below for where these things are
+   used.
+
+   Everything in this phase is time-critical, because it is operating over
+   non-deduplicated types and so may have hundreds or thousands of times the
+   data volume to deal with than later phases.  Trace output is hidden behind
+   ENABLE_LIBCTF_HASH_DEBUGGING to prevent the sheer number of calls to
+   ctf_dprintf from slowing things down (tenfold slowdowns are observed purely
+   from the calls to ctf_dprintf(), even with debugging switched off), and keep
+   down the volume of output (hundreds of gigabytes of debug output are not
+   uncommon on larger links).
+
+   We have to do *something* about potential cycles in the type graph.  We'd
+   like to avoid emitting forwards in the final output if possible, because
+   forwards aren't much use: they have no members.  We are mostly saved from
+   needing to worry about this at emission time by ctf_add_struct*()
+   automatically replacing newly-created forwards when the real struct/union
+   comes along.  So we only have to avoid getting stuck in cycles during the
+   hashing phase, while also not confusing types that cite members that are
+   structs with each other.  It is easiest to solve this problem by noting two
+   things:
+
+    - all cycles in C depend on the presence of tagged structs/unions
+    - all tagged structs/unions have a unique name they can be disambiguated by
+
+   [ctf_dedup_is_stub]
+   This means that we can break all cycles by ceasing to hash in cited types at
+   every tagged struct/union and instead hashing in a stub consisting of the
+   struct/union's *decorated name*, which is the name preceded by "s " or "u "
+   depending on the namespace (cached in cd_decorated_names).  Forwards are
+   decorated identically (so a forward to "struct foo" would be represented as
+   "s foo"): this means that a citation of a forward to a type and a citation of
+   a concrete definition of a type with the same name ends up getting the same
+   hash value.
+
+   Of course, it is quite possible to have two TUs with structs with the same
+   name and different definitions, but that's OK because when we scan for types
+   with ambiguous names we will identify these and mark them conflicting.
+
+   We populate one thing to help conflictedness marking.  No unconflicted type
+   may cite a conflicted one, but this means that conflictedness marking must
+   walk from types to the types that cite them, which is the opposite of the
+   usual order.  We can make this easier to do by constructing a *citers* graph
+   in cd_citers, which points from types to the types that cite them: because we
+   emit forwards corresponding to every conflicted struct/union, we don't need
+   to do this for citations of structs/unions by other types.  This is very
+   convenient for us, because that's the only type we don't traverse
+   recursively: so we can construct the citers graph at the same time as we
+   hash, rather than needing to add an extra pass.  (This graph is a dynhash of
+   *type hash values*, so it's small: in effect it is automatically
+   deduplicated.)
+
+   2) COLLISIONAL MARKING.
+
+   [ctf_dedup_detect_name_ambiguity, ctf_dedup_mark_conflicting_hash]
+   We identify types whose names collide during the hashing process, and count
+   the rough number of uses of each name (caching may throw it off a bit: this
+   doesn't need to be accurate).  We then mark the less-frequently-cited types
+   with each names conflicting: the most-frequently-cited one goes into the
+   shared type dictionary, while all others are duplicated into per-TU
+   dictionaries, named after the input TU, that have the shared dictionary as a
+   parent.  For structures and unions this is not quite good enough: we'd like
+   to have citations of forwards to ambiguously named structures and unions
+   *stay* as citations of forwards, so that the user can tell that the caller
+   didn't actually know which structure definition was meant: but if we put one
+   of those structures into the shared dictionary, it would supplant and replace
+   the forward, leaving no sign.  So structures and unions do not take part in
+   this popularity contest: if their names are ambiguous, they are just
+   duplicated, and only a forward appears in the shared dict.
+
+   [ctf_dedup_propagate_conflictedness]
+   The process of marking types conflicted is itself recursive: we recursively
+   traverse the cd_citers graph populated in the hashing pass above and mark
+   everything that we encounter conflicted (without wasting time re-marking
+   anything that is already marked).  This naturally terminates just where we
+   want it to (at types that are cited by no other types, and at structures and
+   unions) and suffices to ensure that types that cite conflicted types are
+   always marked conflicted.
+
+   [ctf_dedup_conflictify_unshared, ctf_dedup_multiple_input_dicts]
+   When linking in CTF_LINK_SHARE_DUPLICATED mode, we would like all types that
+   are used in only one TU to end up in a per-CU dict. The easiest way to do
+   that is to mark them conflicted.  ctf_dedup_conflictify_unshared does this,
+   traversing the output mapping and using ctf_dedup_multiple_input_dicts to
+   check the number of input dicts each distinct type hash value came from:
+   types that only came from one get marked conflicted.  One caveat here is that
+   we need to consider both structs and forwards to them: a struct that appears
+   in one TU and has a dozen citations to an opaque forward in other TUs should
+   *not* be considered to be used in only one TU, because users would find it
+   useful to be able to traverse into opaque structures of that sort: so we use
+   cd_struct_origin to check both structs/unions and the forwards corresponding
+   to them.
+
+   3) EMISSION.
+
+   [ctf_dedup_walk_output_mapping, ctf_dedup_rwalk_output_mapping,
+    ctf_dedup_rwalk_one_output_mapping]
+   Emission involves another walk of the entire output mapping, this time
+   traversing everything other than struct members, recursively.  Types are
+   emitted from leaves to trunk, emitting all types a type cites before emitting
+   the type itself.  We sort the output mapping before traversing it, for
+   reproducibility and also correctness: the input dicts may have parent/child
+   relationships, so we simply sort all types that first appear in parents
+   before all children, then sort types that first appear in dicts appearing
+   earlier on the linker command line before those that appear later, then sort
+   by input ctf_id_t.  (This is where we use cd_output_first_gid, collected
+   above.)
+
+   The walking is done using a recursive traverser which arranges to not revisit
+   any type already visited and to call its callback once per input GID for
+   input GIDs corresponding to conflicted output types.  The traverser only
+   finds input types and calls a callback for them as many times as the output
+   needs to appear: it doesn't try to figure out anything about where the output
+   might go.  That's done by the callback based on whether the type is
+   marked conflicted or not.
+
+   [ctf_dedup_emit_type, ctf_dedup_id_to_target, ctf_dedup_synthesize_forward]
+   ctf_dedup_emit_type is the (sole) callback for ctf_dedup_walk_output_mapping.
+   Conflicted types have all necessary dictionaries created, and then we emit
+   the type into each dictionary in turn, working over each input CTF type
+   corresponding to each hash value and using ctf_dedup_id_to_target to map each
+   input ctf_id_t into the corresponding type in the output (dealing with input
+   ctf_id_t's with parents in the process by simply chasing to the parent dict
+   if the type we're looking up is in there).  Emitting structures involves
+   simply noting that the members of this structure need emission later on:
+   because you cannot cite a single structure member from another type, we avoid
+   emitting the members at this stage to keep recursion depths down a bit.
+
+   At this point, if we have by some mischance decided that two different types
+   with child types that hash to different values have in fact got the same hash
+   value themselves and *not* marked it conflicting, the type walk will walk
+   only *one* of them and in all likelihood we'll find that we are trying to
+   emit a type into some child dictionary that references a type that was never
+   emitted into that dictionary and assertion-fail.  This always indicates a bug
+   in the conflictedness marking machinery or the hashing code, or both.
+
+   ctf_dedup_id_to_target calls ctf_dedup_synthesize_forward to do one extra
+   thing, alluded to above: if this is a conflicted tagged structure or union,
+   and the target is the shared dict (i.e., the type we're being asked to emit
+   is not itself conflicted so can't just point straight at the conflicted
+   type), we instead synthesise a forward with the same name, emit it into the
+   shared dict, record it in cd_output_emission_conflicted_forwards so that we
+   don't re-emit it, and return it.  This means that cycles that contain
+   conflicts do not cause the entire cycle to be replicated in every child: only
+   that piece of the cycle which takes you back as far as the closest tagged
+   struct/union needs to be replicated.  This trick means that no part of the
+   deduplicator needs a cycle detector: every recursive walk can stop at tagged
+   structures.
+
+   [ctf_dedup_emit_struct_members]
+   The final stage of emission is to walk over all structures with members
+   that need emission and emit all of them. Every type has been emitted at
+   this stage, so emission cannot fail.
+
+   [ctf_dedup_populate_type_mappings, ctf_dedup_populate_type_mapping]
+   Finally, we update the input -> output type ID mappings used by the ctf-link
+   machinery to update all the other sections.  This is surprisingly expensive
+   and may be replaced with a scheme which lets the ctf-link machinery extract
+   the needed info directly from the deduplicator.  */
+
+/* Possible future optimizations are flagged with 'optimization opportunity'
+   below.  */
+
+/* Global optimization opportunity: a GC pass, eliminating types with no direct
+   or indirect citations from the other sections in the dictionary.  */
+
+/* Internal flag values for ctf_dedup_hash_type.  */
+
+/* Child call: consider forwardable types equivalent to forwards or stubs below
+   this point.  */
+#define CTF_DEDUP_HASH_INTERNAL_CHILD         0x01
+
+/* Transform references to single ctf_id_ts in passed-in inputs into a number
+   that will fit in a uint64_t.  Needs rethinking if CTF_MAX_TYPE is boosted.
+
+   On 32-bit platforms, we pack things together differently: see the note
+   above.  */
+
+#if UINTPTR_MAX < UINT64_MAX
+# define IDS_NEED_ALLOCATION 1
+# define CTF_DEDUP_GID(fp, input, type) id_to_packed_id (fp, input, type)
+# define CTF_DEDUP_GID_TO_INPUT(id) packed_id_to_input (id)
+# define CTF_DEDUP_GID_TO_TYPE(id) packed_id_to_type (id)
+#else
+# define CTF_DEDUP_GID(fp, input, type)	\
+  (void *) (((uint64_t) input) << 32 | (type))
+# define CTF_DEDUP_GID_TO_INPUT(id) ((int) (((uint64_t) id) >> 32))
+# define CTF_DEDUP_GID_TO_TYPE(id) (ctf_id_t) (((uint64_t) id) & ~(0xffffffff00000000ULL))
+#endif
+
+#ifdef IDS_NEED_ALLOCATION
+
+ /* This is the 32-bit path, which stores GIDs in a pool and returns a pointer
+    into the pool.  It is notably less efficient than the 64-bit direct storage
+    approach, but with a smaller key, this is all we can do.  */
+
+static void *
+id_to_packed_id (ctf_file_t *fp, int input_num, ctf_id_t type)
+{
+  const void *lookup;
+  ctf_type_id_key_t *dynkey = NULL;
+  ctf_type_id_key_t key = { input_num, type };
+
+  if (!ctf_dynhash_lookup_kv (fp->ctf_dedup.cd_id_to_file_t,
+			      &key, &lookup, NULL))
+    {
+      if ((dynkey = malloc (sizeof (ctf_type_id_key_t))) == NULL)
+	goto oom;
+      memcpy (dynkey, &key, sizeof (ctf_type_id_key_t));
+
+      if (ctf_dynhash_insert (fp->ctf_dedup.cd_id_to_file_t, dynkey, NULL) < 0)
+	goto oom;
+
+      ctf_dynhash_lookup_kv (fp->ctf_dedup.cd_id_to_file_t,
+			     dynkey, &lookup, NULL);
+    }
+  /* We use a raw assert() here because there isn't really a way to get any sort
+     of error back from this routine without vastly complicating things for the
+     much more common case of !IDS_NEED_ALLOCATION.  */
+  assert (lookup);
+  return (void *) lookup;
+
+ oom:
+  free (dynkey);
+  ctf_set_errno (fp, ENOMEM);
+  return NULL;
+}
+
+static int
+packed_id_to_input (const void *id)
+{
+  const ctf_type_id_key_t *key = (ctf_type_id_key_t *) id;
+
+  return key->ctii_input_num;
+}
+
+static ctf_id_t
+packed_id_to_type (const void *id)
+{
+  const ctf_type_id_key_t *key = (ctf_type_id_key_t *) id;
+
+  return key->ctii_type;
+}
+#endif
+
+/* Make an element in a dynhash-of-dynsets, or return it if already present.  */
+
+static ctf_dynset_t *
+make_set_element (ctf_dynhash_t *set, const void *key)
+{
+  ctf_dynset_t *element;
+
+  if ((element = ctf_dynhash_lookup (set, key)) == NULL)
+    {
+      if ((element = ctf_dynset_create (htab_hash_string,
+					ctf_dynset_eq_string,
+					NULL)) == NULL)
+	return NULL;
+
+      if (ctf_dynhash_insert (set, (void *) key, element) < 0)
+	{
+	  ctf_dynset_destroy (element);
+	  return NULL;
+	}
+    }
+
+  return element;
+}
+
+/* Initialize the dedup atoms table.  */
+int
+ctf_dedup_atoms_init (ctf_file_t *fp)
+{
+  if (fp->ctf_dedup_atoms)
+    return 0;
+
+  if (!fp->ctf_dedup_atoms_alloc)
+    {
+      if ((fp->ctf_dedup_atoms_alloc
+	   = ctf_dynset_create (htab_hash_string, ctf_dynset_eq_string,
+				free)) == NULL)
+	return ctf_set_errno (fp, ENOMEM);
+    }
+  fp->ctf_dedup_atoms = fp->ctf_dedup_atoms_alloc;
+  return 0;
+}
+
+/* Intern things in the dedup atoms table.  */
+
+static const char *
+intern (ctf_file_t *fp, char *atom)
+{
+  const void *foo;
+
+  if (atom == NULL)
+    return NULL;
+
+  if (!ctf_dynset_exists (fp->ctf_dedup_atoms, atom, &foo))
+    {
+      if (ctf_dynset_insert (fp->ctf_dedup_atoms, atom) < 0)
+	{
+	  ctf_set_errno (fp, ENOMEM);
+	  return NULL;
+	}
+      foo = atom;
+    }
+  else
+    free (atom);
+
+  return (const char *) foo;
+}
+
+/* Add an indication of the namespace to a type name in a way that is not valid
+   for C identifiers.  Used to maintain hashes of type names to other things
+   while allowing for the four C namespaces (normal, struct, union, enum).
+   Return a new dynamically-allocated string.  */
+static const char *
+ctf_decorate_type_name (ctf_file_t *fp, const char *name, int kind)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  const char *ret;
+  const char *k;
+  char *p;
+  size_t i;
+
+  switch (kind)
+    {
+    case CTF_K_STRUCT:
+      k = "s ";
+      i = 0;
+      break;
+    case CTF_K_UNION:
+      k = "u ";
+      i = 1;
+      break;
+    case CTF_K_ENUM:
+      k = "e ";
+      i = 2;
+      break;
+    default:
+      k = "";
+      i = 3;
+    }
+
+  if ((ret = ctf_dynhash_lookup (d->cd_decorated_names[i], name)) == NULL)
+    {
+      char *str;
+
+      if ((str = malloc (strlen (name) + strlen (k) + 1)) == NULL)
+	goto oom;
+
+      p = stpcpy (str, k);
+      strcpy (p, name);
+      ret = intern (fp, str);
+      if (!ret)
+	goto oom;
+
+      if (ctf_dynhash_cinsert (d->cd_decorated_names[i], name, ret) < 0)
+	goto oom;
+    }
+
+  return ret;
+
+ oom:
+  ctf_set_errno (fp, ENOMEM);
+  return NULL;
+}
+
+/* Hash a type, possibly debugging-dumping something about it as well.  */
+static inline void
+ctf_dedup_sha1_add (ctf_sha1_t *sha1, const void *buf, size_t len,
+		    const char *description _libctf_unused_,
+		    unsigned long depth _libctf_unused_)
+{
+  ctf_sha1_add (sha1, buf, len);
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_sha1_t tmp;
+  char tmp_hval[CTF_SHA1_SIZE];
+  tmp = *sha1;
+  ctf_sha1_fini (&tmp, tmp_hval);
+  ctf_dprintf ("%lu: after hash addition of %s: %s\n", depth, description,
+	       tmp_hval);
+#endif
+}
+
+static const char *
+ctf_dedup_hash_type (ctf_file_t *fp, ctf_file_t *input,
+		     ctf_file_t **inputs, uint32_t *parents,
+		     int input_num, ctf_id_t type, int flags,
+		     unsigned long depth,
+		     int (*populate_fun) (ctf_file_t *fp,
+					  ctf_file_t *input,
+					  ctf_file_t **inputs,
+					  int input_num,
+					  ctf_id_t type,
+					  void *id,
+					  const char *decorated_name,
+					  const char *hash));
+
+/* Determine whether this type is being hashed as a stub (in which case it is
+   unsafe to cache it).  */
+static int
+ctf_dedup_is_stub (const char *name, int kind, int fwdkind, int flags)
+{
+  /* We can cache all types unless we are recursing to children and are hashing
+     in a tagged struct, union or forward, all of which are replaced with their
+     decorated name as a stub and will have different hash values when hashed at
+     the top level.  */
+
+  return ((flags & CTF_DEDUP_HASH_INTERNAL_CHILD) && name
+	  && (kind == CTF_K_STRUCT || kind == CTF_K_UNION
+	      || (kind == CTF_K_FORWARD && (fwdkind == CTF_K_STRUCT
+					    || fwdkind == CTF_K_UNION))));
+}
+
+/* Populate struct_origin if need be (not already populated, or populated with
+   a different origin), in which case it must go to -1, "shared".)
+
+   Only called for forwards or forwardable types with names, when the link mode
+   is CTF_LINK_SHARE_DUPLICATED.  */
+static int
+ctf_dedup_record_origin (ctf_file_t *fp, int input_num, const char *decorated,
+			 void *id)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  void *origin;
+  int populate_origin = 0;
+
+  if (ctf_dynhash_lookup_kv (d->cd_struct_origin, decorated, NULL, &origin))
+    {
+      if (CTF_DEDUP_GID_TO_INPUT (origin) != input_num
+	  && CTF_DEDUP_GID_TO_INPUT (origin) != -1)
+	{
+	  populate_origin = 1;
+	  origin = CTF_DEDUP_GID (fp, -1, -1);
+	}
+    }
+  else
+    {
+      populate_origin = 1;
+      origin = id;
+    }
+
+  if (populate_origin)
+    if (ctf_dynhash_cinsert (d->cd_struct_origin, decorated, origin) < 0)
+      return ctf_set_errno (fp, errno);
+  return 0;
+}
+
+/* Do the underlying hashing and recursion for ctf_dedup_hash_type (which it
+   calls, recursively).  */
+
+static const char *
+ctf_dedup_rhash_type (ctf_file_t *fp, ctf_file_t *input, ctf_file_t **inputs,
+		      uint32_t *parents, int input_num, ctf_id_t type,
+		      void *type_id, const ctf_type_t *tp, const char *name,
+		      const char *decorated, int kind, int flags,
+		      unsigned long depth,
+		      int (*populate_fun) (ctf_file_t *fp,
+					   ctf_file_t *input,
+					   ctf_file_t **inputs,
+					   int input_num,
+					   ctf_id_t type,
+					   void *id,
+					   const char *decorated_name,
+					   const char *hash))
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  ctf_next_t *i = NULL;
+  ctf_sha1_t hash;
+  ctf_id_t child_type;
+  char hashbuf[CTF_SHA1_SIZE];
+  const char *hval = NULL;
+  const char *whaterr;
+  int err;
+
+  const char *citer = NULL;
+  ctf_dynset_t *citers = NULL;
+
+  /* Add a citer to the citers set.  */
+#define ADD_CITER(citers, hval)						\
+  do									\
+    {									\
+      whaterr = "updating citers";					\
+      if (!citers)							\
+	if ((citers = ctf_dynset_create (htab_hash_string,		\
+					  ctf_dynset_eq_string,		\
+					  NULL)) == NULL)		\
+	  goto oom;							\
+      if (ctf_dynset_cinsert (citers, hval) < 0)			\
+	goto oom;							\
+    } while (0)
+
+  /* If this is a named struct or union or a forward to one, and this is a child
+     traversal, treat this type as if it were a forward -- do not recurse to
+     children, ignore all content not already hashed in, and hash in the
+     decorated name of the type instead.  */
+
+  if (ctf_dedup_is_stub (name, kind, tp->ctt_type, flags))
+    {
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+      ctf_dprintf ("Struct/union/forward citation: substituting forwarding "
+		   "stub with decorated name %s\n", decorated);
+
+#endif
+      ctf_sha1_init (&hash);
+      ctf_dedup_sha1_add (&hash, decorated, strlen (decorated) + 1,
+			  "decorated struct/union/forward name", depth);
+      ctf_sha1_fini (&hash, hashbuf);
+
+      if ((hval = intern (fp, strdup (hashbuf))) == NULL)
+	{
+	  ctf_err_warn (fp, 0, "%s (%i): out of memory during forwarding-stub "
+			"hashing for type with GID %p; errno: %s",
+			ctf_link_input_name (input), input_num, type_id,
+			strerror (errno));
+	  return NULL;				/* errno is set for us.  */
+	}
+
+      /* In share--duplicated link mode, make sure the origin of this type is
+	 recorded, even if this is a type in a parent dict which will not be
+	 directly traversed.  */
+      if (d->cd_link_flags & CTF_LINK_SHARE_DUPLICATED
+	  && ctf_dedup_record_origin (fp, input_num, decorated, type_id) < 0)
+	return NULL;				/* errno is set for us.  */
+
+      return hval;
+    }
+
+  /* Now ensure that subsequent recursive calls (but *not* the top-level call)
+     get this treatment.  */
+  flags |= CTF_DEDUP_HASH_INTERNAL_CHILD;
+
+  /* If this is a struct, union, or forward with a name, record the unique
+     originating input TU, if there is one.  */
+
+  if (decorated && (ctf_forwardable_kind (kind) || kind != CTF_K_FORWARD))
+    if (d->cd_link_flags & CTF_LINK_SHARE_DUPLICATED
+	&& ctf_dedup_record_origin (fp, input_num, decorated, type_id) < 0)
+      return NULL;				/* errno is set for us.  */
+
+  /* Mix in invariant stuff, transforming the type kind if needed.  Note that
+     the vlen is *not* hashed in: the actual variable-length info is hashed in
+     instead, piecewise.  The vlen is not part of the type, only the
+     variable-length data is: identical types with distinct vlens are quite
+     possible.  Equally, we do not want to hash in the isroot flag: both the
+     compiler and the deduplicator set the nonroot flag to indicate clashes with
+     *other types in the same TU* with the same name: so two types can easily
+     have distinct nonroot flags, yet be exactly the same type.*/
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("%lu: hashing thing with ID %i/%lx (kind %i): %s.\n",
+	       depth, input_num, type, kind, name ? name : "");
+#endif
+
+  ctf_sha1_init (&hash);
+  if (name)
+    ctf_dedup_sha1_add (&hash, name, strlen (name) + 1, "name", depth);
+  ctf_dedup_sha1_add (&hash, &kind, sizeof (uint32_t), "kind", depth);
+
+  /* Hash content of this type.  */
+  switch (kind)
+    {
+    case CTF_K_UNKNOWN:
+      /* No extra state.  */
+      break;
+    case CTF_K_FORWARD:
+
+      /* Add the forwarded kind, stored in the ctt_type.  */
+      ctf_dedup_sha1_add (&hash, &tp->ctt_type, sizeof (tp->ctt_type),
+			  "forwarded kind", depth);
+      break;
+    case CTF_K_INTEGER:
+    case CTF_K_FLOAT:
+      {
+	ctf_encoding_t ep;
+	memset (&ep, 0, sizeof (ctf_encoding_t));
+
+	ctf_dedup_sha1_add (&hash, &tp->ctt_size, sizeof (uint32_t), "size",
+			    depth);
+	if (ctf_type_encoding (input, type, &ep) < 0)
+	  {
+	    whaterr = "encoding";
+	    goto err;
+	  }
+	ctf_dedup_sha1_add (&hash, &ep, sizeof (ctf_encoding_t), "encoding",
+			    depth);
+	break;
+      }
+      /* Types that reference other types.  */
+    case CTF_K_TYPEDEF:
+    case CTF_K_VOLATILE:
+    case CTF_K_CONST:
+    case CTF_K_RESTRICT:
+    case CTF_K_POINTER:
+      /* Hash the referenced type, if not already hashed, and mix it in.  */
+      child_type = ctf_type_reference (input, type);
+      if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents, input_num,
+				       child_type, flags, depth,
+				       populate_fun)) == NULL)
+	{
+	  whaterr = "referenced type hashing";
+	  goto err;
+	}
+      ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "referenced type",
+			  depth);
+      citer = hval;
+
+      break;
+
+      /* The slices of two types hash identically only if the type they overlay
+	 also has the same encoding.  This is not ideal, but in practice will work
+	 well enough.  We work directly rather than using the CTF API because
+	 we do not want the slice's normal automatically-shine-through
+	 semantics to kick in here.  */
+    case CTF_K_SLICE:
+      {
+	const ctf_slice_t *slice;
+	const ctf_dtdef_t *dtd;
+	ssize_t size;
+	ssize_t increment;
+
+	child_type = ctf_type_reference (input, type);
+	ctf_get_ctt_size (input, tp, &size, &increment);
+	ctf_dedup_sha1_add (&hash, &size, sizeof (ssize_t), "size", depth);
+
+	if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents, input_num,
+					 child_type, flags, depth,
+					 populate_fun)) == NULL)
+	  {
+	    whaterr = "slice-referenced type hashing";
+	    goto err;
+	  }
+	ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "sliced type",
+			    depth);
+	citer = hval;
+
+	if ((dtd = ctf_dynamic_type (input, type)) != NULL)
+	  slice = &dtd->dtd_u.dtu_slice;
+	else
+	  slice = (ctf_slice_t *) ((uintptr_t) tp + increment);
+
+	ctf_dedup_sha1_add (&hash, &slice->cts_offset,
+			    sizeof (slice->cts_offset), "slice offset", depth);
+	ctf_dedup_sha1_add (&hash, &slice->cts_bits,
+			    sizeof (slice->cts_bits), "slice bits", depth);
+	break;
+      }
+
+    case CTF_K_ARRAY:
+      {
+	ctf_arinfo_t ar;
+
+	if (ctf_array_info (input, type, &ar) < 0)
+	  {
+	    whaterr = "array info";
+	    goto err;
+	  }
+
+	if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents, input_num,
+					 ar.ctr_contents, flags, depth,
+					 populate_fun)) == NULL)
+	  {
+	    whaterr = "array contents type hashing";
+	    goto err;
+	  }
+	ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "array contents",
+			    depth);
+	ADD_CITER (citers, hval);
+
+	if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents, input_num,
+					 ar.ctr_index, flags, depth,
+					 populate_fun)) == NULL)
+	  {
+	    whaterr = "array index type hashing";
+	    goto err;
+	  }
+	ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "array index",
+			    depth);
+	ctf_dedup_sha1_add (&hash, &ar.ctr_nelems, sizeof (ar.ctr_nelems),
+			    "element count", depth);
+	ADD_CITER (citers, hval);
+
+	break;
+      }
+    case CTF_K_FUNCTION:
+      {
+	ctf_funcinfo_t fi;
+	ctf_id_t *args;
+	uint32_t j;
+
+	if (ctf_func_type_info (input, type, &fi) < 0)
+	  {
+	    whaterr = "func type info";
+	    goto err;
+	  }
+
+	if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents, input_num,
+					 fi.ctc_return, flags, depth,
+					 populate_fun)) == NULL)
+	  {
+	    whaterr = "func return type";
+	    goto err;
+	  }
+	ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "func return",
+			    depth);
+	ctf_dedup_sha1_add (&hash, &fi.ctc_argc, sizeof (fi.ctc_argc),
+			    "func argc", depth);
+	ctf_dedup_sha1_add (&hash, &fi.ctc_flags, sizeof (fi.ctc_flags),
+			    "func flags", depth);
+	ADD_CITER (citers, hval);
+
+	if ((args = calloc (fi.ctc_argc, sizeof (ctf_id_t))) == NULL)
+	  {
+	    whaterr = "memory allocation";
+	    goto err;
+	  }
+
+	if (ctf_func_type_args (input, type, fi.ctc_argc, args) < 0)
+	  {
+	    free (args);
+	    whaterr = "func arg type";
+	    goto err;
+	  }
+	for (j = 0; j < fi.ctc_argc; j++)
+	  {
+	    if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents,
+					     input_num, args[j], flags, depth,
+					     populate_fun)) == NULL)
+	      {
+		free (args);
+		whaterr = "func arg type hashing";
+		goto err;
+	      }
+	    ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "func arg type",
+				depth);
+	    ADD_CITER (citers, hval);
+	  }
+	free (args);
+	break;
+      }
+    case CTF_K_ENUM:
+      {
+	int val;
+	const char *ename;
+
+	ctf_dedup_sha1_add (&hash, &tp->ctt_size, sizeof (uint32_t),
+			    "enum size", depth);
+	while ((ename = ctf_enum_next (input, type, &i, &val)) != NULL)
+	  {
+	    ctf_dedup_sha1_add (&hash, ename, strlen (ename) + 1, "enumerator",
+				depth);
+	    ctf_dedup_sha1_add (&hash, &val, sizeof (val), "enumerand", depth);
+	  }
+	if (ctf_errno (input) != ECTF_NEXT_END)
+	  {
+	    whaterr = "enum member iteration";
+	    goto err;
+	  }
+	break;
+      }
+    /* Top-level only.  */
+    case CTF_K_STRUCT:
+    case CTF_K_UNION:
+      {
+	ssize_t offset;
+	const char *mname;
+	ctf_id_t membtype;
+	ssize_t size;
+
+	ctf_get_ctt_size (input, tp, &size, NULL);
+	ctf_dedup_sha1_add (&hash, &size, sizeof (ssize_t), "struct size",
+			    depth);
+
+	while ((offset = ctf_member_next (input, type, &i, &mname,
+					  &membtype)) >= 0)
+	  {
+	    if (mname == NULL)
+	      mname = "";
+	    ctf_dedup_sha1_add (&hash, mname, strlen (mname) + 1,
+				"member name", depth);
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+	    ctf_dprintf ("%lu: Traversing to member %s\n", depth, mname);
+#endif
+	    if ((hval = ctf_dedup_hash_type (fp, input, inputs, parents,
+					     input_num, membtype, flags, depth,
+					     populate_fun)) == NULL)
+	      {
+		whaterr = "struct/union member type hashing";
+		goto iterr;
+	      }
+
+	    ctf_dedup_sha1_add (&hash, hval, strlen (hval) + 1, "member hash",
+				depth);
+	    ctf_dedup_sha1_add (&hash, &offset, sizeof (offset), "member offset",
+				depth);
+	    ADD_CITER (citers, hval);
+	  }
+	if (ctf_errno (input) != ECTF_NEXT_END)
+	  {
+	    whaterr = "struct/union member iteration";
+	    goto err;
+	  }
+	break;
+      }
+    default:
+      whaterr = "unknown type kind";
+      goto err;
+    }
+  ctf_sha1_fini (&hash, hashbuf);
+
+  if ((hval = intern (fp, strdup (hashbuf))) == NULL)
+    {
+      whaterr = "hash internment";
+      goto oom;
+    }
+
+  /* Populate the citers for this type's subtypes, now the hash for the type
+     itself is known.  */
+  whaterr = "citer tracking";
+
+  if (citer)
+    {
+      ctf_dynset_t *citer_hashes;
+
+      if ((citer_hashes = make_set_element (d->cd_citers, citer)) == NULL)
+	goto oom;
+      if (ctf_dynset_cinsert (citer_hashes, hval) < 0)
+	goto oom;
+    }
+  else if (citers)
+    {
+      const void *k;
+
+      while ((err = ctf_dynset_cnext (citers, &i, &k)) == 0)
+	{
+	  ctf_dynset_t *citer_hashes;
+	  citer = (const char *) k;
+
+	  if ((citer_hashes = make_set_element (d->cd_citers, citer)) == NULL)
+	    goto oom;
+
+	  if (ctf_dynset_exists (citer_hashes, hval, NULL))
+	    continue;
+	  if (ctf_dynset_cinsert (citer_hashes, hval) < 0)
+	    goto oom;
+	}
+      if (err != ECTF_NEXT_END)
+	goto err;
+      ctf_dynset_destroy (citers);
+    }
+
+  return hval;
+
+ iterr:
+  ctf_next_destroy (i);
+ err:
+  ctf_sha1_fini (&hash, NULL);
+  ctf_err_warn (fp, 0, "%s (%i): %s error during type hashing for "
+		"type %lx, kind %i: CTF error: %s; errno: %s",
+		ctf_link_input_name (input), input_num, whaterr, type,
+		kind, ctf_errmsg (ctf_errno (fp)), strerror (errno));
+  return NULL;
+ oom:
+  ctf_set_errno (fp, errno);
+  ctf_err_warn (fp, 0, "%s (%i): %s error during type hashing for "
+		"type %lx, kind %i: CTF error: %s; errno: %s",
+		ctf_link_input_name (input), input_num, whaterr, type,
+		kind, ctf_errmsg (ctf_errno (fp)), strerror (errno));
+  return NULL;
+}
+
+/* Internal implementation of ctf_dedup_hash_type (below), which is the entry
+   point others should call.
+
+   Hash a TYPE in the INPUT: FP is the eventual output, where the ctf_dedup
+   state is stored.  INPUT_NUM is the number of this input in the set of inputs.
+   Record its hash in FP's cd_type_hashes once it is known.  PARENTS is
+   described in the comment above ctf_dedup.
+
+   (The flags argument currently accepts only the flag
+   CTF_DEDUP_HASH_INTERNAL_CHILD, an implementation detail used to prevent
+   struct/union hashing in recursive traversals below the TYPE.)
+
+   We use the CTF API rather than direct access wherever possible, because types
+   that appear identical through the API should be considered identical, with
+   one exception: slices should only be considered identical to other slices,
+   not to the corresponding unsliced type.
+
+   The POPULATE_FUN is a mandatory hook that populates other mappings with each
+   type we see (excepting types that are recursively hashed as stubs).  The
+   caller should not rely on the order of calls to this hook, though it will be
+   called at least once for every non-stub reference to every type.
+
+   Returns a hash value (an atom), or NULL on error.  */
+
+static const char *
+ctf_dedup_hash_type (ctf_file_t *fp, ctf_file_t *input,
+		     ctf_file_t **inputs, uint32_t *parents,
+		     int input_num, ctf_id_t type, int flags,
+		     unsigned long depth,
+		     int (*populate_fun) (ctf_file_t *fp,
+					  ctf_file_t *input,
+					  ctf_file_t **inputs,
+					  int input_num,
+					  ctf_id_t type,
+					  void *id,
+					  const char *decorated_name,
+					  const char *hash))
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  const ctf_type_t *tp;
+  void *type_id;
+  const char *hval = NULL;
+  const char *name;
+  const char *whaterr;
+  const char *decorated = NULL;
+  uint32_t kind, fwdkind;
+
+  depth++;
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("%lu: ctf_dedup_hash_type (%i, %lx, flags %x)\n", depth, input_num, type, flags);
+#endif
+
+  /* The unimplemented type doesn't really exist, but must be noted in parent
+     hashes: so it gets a fixed, arbitrary hash.  */
+  if (type == 0)
+    return "00000000000000000000";
+
+  /* Possible optimization: if the input type is in the parent type space, just
+     copy recursively-cited hashes from the parent's types into the output
+     mapping rather than rehashing them.  */
+
+  type_id = CTF_DEDUP_GID (fp, input_num, type);
+
+  if ((tp = ctf_lookup_by_id (&input, type)) == NULL)
+    {
+      ctf_err_warn (fp, 0, "%s (%i): lookup failure for type %lx: flags %x",
+		    ctf_link_input_name (input), input_num, type, flags);
+      return NULL;		/* errno is set for us.  */
+    }
+
+  kind = LCTF_INFO_KIND (input, tp->ctt_info);
+  name = ctf_strraw (input, tp->ctt_name);
+
+  if (tp->ctt_name == 0 || !name || name[0] == '\0')
+    name = NULL;
+
+  /* Treat the unknown kind just like the unimplemented type.  */
+  if (kind == CTF_K_UNKNOWN)
+    return "00000000000000000000";
+
+  /* Decorate the name appropriately for the namespace it appears in: forwards
+     appear in the namespace of their referent.  */
+
+  fwdkind = kind;
+  if (name)
+    {
+      if (kind == CTF_K_FORWARD)
+	fwdkind = tp->ctt_type;
+
+      if ((decorated = ctf_decorate_type_name (fp, name, fwdkind)) == NULL)
+	return NULL;				/* errno is set for us.  */
+    }
+
+  /* If not hashing a stub, we can rely on various sorts of caches.
+
+     Optimization opportunity: we may be able to avoid calling the populate_fun
+     sometimes here.  */
+
+  if (!ctf_dedup_is_stub (name, kind, fwdkind, flags))
+    {
+      if ((hval = ctf_dynhash_lookup (d->cd_type_hashes, type_id)) != NULL)
+	{
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+	  ctf_dprintf ("%lu: Known hash for ID %i/%lx: %s\n", depth, input_num,
+		       type,  hval);
+#endif
+	  populate_fun (fp, input, inputs, input_num, type, type_id,
+			decorated, hval);
+
+	  return hval;
+	}
+    }
+
+  /* We have never seen this type before, and must figure out its hash and the
+     hashes of the types it cites.
+
+     Hash this type, and call ourselves recursively.  (The hashing part is
+     optional, and is disabled if overidden_hval is set.)  */
+
+  if ((hval = ctf_dedup_rhash_type (fp, input, inputs, parents, input_num,
+				    type, type_id, tp, name, decorated,
+				    kind, flags, depth, populate_fun)) == NULL)
+    return NULL;				/* errno is set for us.  */
+
+  /* The hash of this type is now known: record it unless caching is unsafe
+     because the hash value will change later.  This will be the final storage
+     of this type's hash, so we call the population function on it.  */
+
+  if (!ctf_dedup_is_stub (name, kind, fwdkind, flags))
+    {
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+      ctf_dprintf ("Caching %lx, ID %p (%s), %s in final location\n", type,
+		   type_id, name ? name : "", hval);
+#endif
+
+      if (ctf_dynhash_cinsert (d->cd_type_hashes, type_id, hval) < 0)
+	{
+	  whaterr = "hash caching";
+	  goto oom;
+	}
+
+      if (populate_fun (fp, input, inputs, input_num, type, type_id,
+			decorated, hval) < 0)
+	{
+	  whaterr = "population function";
+	  goto err;				/* errno is set for us. */
+	}
+    }
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("%lu: Returning final hash for ID %i/%lx: %s\n", depth,
+	       input_num, type, hval);
+#endif
+  return hval;
+
+ oom:
+  ctf_set_errno (fp, errno);
+ err:
+  ctf_err_warn (fp, 0, "%s (%i): %s error during type hashing, type %lx, "
+		"kind %i: CTF errno: %s; errno: %s",
+		ctf_link_input_name (input), input_num, whaterr, type,
+		kind, ctf_errmsg (ctf_errno (fp)),
+		strerror (errno));
+  return NULL;
+}
+
+/* Populate a number of useful mappings not directly used by the hashing
+   machinery: the output mapping, the cd_name_counts mapping from name -> hash
+   -> count of hashval deduplication state for a given hashed type, and the
+   cd_output_first_tu mapping.  */
+
+static int
+ctf_dedup_populate_mappings (ctf_file_t *fp, ctf_file_t *input _libctf_unused_,
+			     ctf_file_t **inputs _libctf_unused_,
+			     int input_num _libctf_unused_,
+			     ctf_id_t type _libctf_unused_, void *id,
+			     const char *decorated_name,
+			     const char *hval)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  ctf_dynset_t *type_ids;
+  ctf_dynhash_t *name_counts;
+  long int count;
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dprintf ("Hash %s, %s, into output mapping for %i/%lx @ %s\n",
+	       hval, decorated_name ? decorated_name : "(unnamed)",
+	       input_num, type, ctf_link_input_name (input));
+
+  const char *orig_hval;
+
+  /* Make sure we never map a single GID to multiple hash values.  */
+
+  if ((orig_hval = ctf_dynhash_lookup (d->cd_output_mapping_guard, id)) != NULL)
+    {
+      /* We can rely on pointer identity here, since all hashes are
+	 interned.  */
+      if (!ctf_assert (fp, orig_hval == hval))
+	return -1;
+    }
+  else
+    if (ctf_dynhash_cinsert (d->cd_output_mapping_guard, id, hval) < 0)
+      return ctf_set_errno (fp, errno);
+#endif
+
+  /* Record the type in the output mapping: if this is the first time this type
+     has been seen, also record it in the cd_output_first_gid.  Because we
+     traverse types in TU order and we do not merge types after the hashing
+     phase, this will be the lowest TU this type ever appears in.  */
+
+  if ((type_ids = ctf_dynhash_lookup (d->cd_output_mapping,
+				      hval)) == NULL)
+    {
+      if (ctf_dynhash_cinsert (d->cd_output_first_gid, hval, id) < 0)
+	return ctf_set_errno (fp, errno);
+
+      if ((type_ids = ctf_dynset_create (htab_hash_pointer,
+					 htab_eq_pointer,
+					 NULL)) == NULL)
+	return ctf_set_errno (fp, errno);
+      if (ctf_dynhash_insert (d->cd_output_mapping, (void *) hval,
+			      type_ids) < 0)
+	{
+	  ctf_dynset_destroy (type_ids);
+	  return ctf_set_errno (fp, errno);
+	}
+    }
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+    {
+      /* Verify that all types with this hash are of the same kind, and that the
+	 first TU a type was seen in never falls.  */
+
+      int err;
+      const void *one_id;
+      ctf_next_t *i = NULL;
+      int orig_kind = ctf_type_kind_unsliced (input, type);
+      int orig_first_tu;
+
+      orig_first_tu = CTF_DEDUP_GID_TO_INPUT
+	(ctf_dynhash_lookup (d->cd_output_first_gid, hval));
+      if (!ctf_assert (fp, orig_first_tu <= CTF_DEDUP_GID_TO_INPUT (id)))
+	return -1;
+
+      while ((err = ctf_dynset_cnext (type_ids, &i, &one_id)) == 0)
+	{
+	  ctf_file_t *foo = inputs[CTF_DEDUP_GID_TO_INPUT (one_id)];
+	  ctf_id_t bar = CTF_DEDUP_GID_TO_TYPE (one_id);
+	  if (ctf_type_kind_unsliced (foo, bar) != orig_kind)
+	    {
+	      ctf_err_warn (fp, 1, "added wrong kind to output mapping "
+			    "for hash %s named %s: %p/%lx from %s is "
+			    "kind %i, but newly-added %p/%lx from %s is "
+			    "kind %i", hval,
+			    decorated_name ? decorated_name : "(unnamed)",
+			    (void *) foo, bar,
+			    ctf_link_input_name (foo),
+			    ctf_type_kind_unsliced (foo, bar),
+			    (void *) input, type,
+			    ctf_link_input_name (input), orig_kind);
+	      if (!ctf_assert (fp, ctf_type_kind_unsliced (foo, bar)
+			       == orig_kind))
+		return -1;
+	    }
+	}
+      if (err != ECTF_NEXT_END)
+	return ctf_set_errno (fp, err);
+    }
+#endif
+
+  /* This function will be repeatedly called for the same types many times:
+     don't waste time reinserting the same keys in that case.  */
+  if (!ctf_dynset_exists (type_ids, id, NULL)
+      && ctf_dynset_insert (type_ids, id) < 0)
+    return ctf_set_errno (fp, errno);
+
+  /* The rest only needs to happen for types with names.  */
+  if (!decorated_name)
+    return 0;
+
+  /* Count the number of occurrences of the hash value for this GID.  */
+
+  hval = ctf_dynhash_lookup (d->cd_type_hashes, id);
+
+  /* Mapping from name -> hash(hashval, count) not already present?  */
+  if ((name_counts = ctf_dynhash_lookup (d->cd_name_counts,
+					 decorated_name)) == NULL)
+    {
+      if ((name_counts = ctf_dynhash_create (ctf_hash_string,
+					     ctf_hash_eq_string,
+					     NULL, NULL)) == NULL)
+	  return ctf_set_errno (fp, errno);
+      if (ctf_dynhash_cinsert (d->cd_name_counts, decorated_name,
+			       name_counts) < 0)
+	{
+	  ctf_dynhash_destroy (name_counts);
+	  return ctf_set_errno (fp, errno);
+	}
+    }
+
+  /* This will, conveniently, return NULL (i.e. 0) for a new entry.  */
+  count = (long int) ctf_dynhash_lookup (name_counts, hval);
+
+  if (ctf_dynhash_cinsert (name_counts, hval,
+			  (const void *) (count + 1)) < 0)
+    return ctf_set_errno (fp, errno);
+
+  return 0;
+}
+
+/* Mark a single hash as corresponding to a conflicting type.  Mark all types
+   that cite it as conflicting as well, terminating the recursive walk only when
+   types that are already conflicted or types do not cite other types are seen.
+   (Tagged structures and unions do not appear in the cd_citers graph, so the
+   walk also terminates there, since any reference to a conflicting structure is
+   just going to reference an unconflicting forward instead: see
+   ctf_dedup_maybe_synthesize_forward.)  */
+
+static int
+ctf_dedup_mark_conflicting_hash (ctf_file_t *fp, const char *hval)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  ctf_next_t *i = NULL;
+  int err;
+  const void *k;
+  ctf_dynset_t *citers;
+
+  /* Mark conflicted if not already so marked.  */
+  if (ctf_dynset_exists (d->cd_conflicting_types, hval, NULL))
+    return 0;
+
+  ctf_dprintf ("Marking %s as conflicted\n", hval);
+
+  if (ctf_dynset_cinsert (d->cd_conflicting_types, hval) < 0)
+    {
+      ctf_dprintf ("Out of memory marking %s as conflicted\n", hval);
+      ctf_set_errno (fp, errno);
+      return -1;
+    }
+
+  /* If any types cite this type, mark them conflicted too.  */
+  if ((citers = ctf_dynhash_lookup (d->cd_citers, hval)) == NULL)
+    return 0;
+
+  while ((err = ctf_dynset_cnext (citers, &i, &k)) == 0)
+    {
+      const char *hv = (const char *) k;
+
+      if (ctf_dynset_exists (d->cd_conflicting_types, hv, NULL))
+	continue;
+
+      if (ctf_dedup_mark_conflicting_hash (fp, hv) < 0)
+	{
+	  ctf_next_destroy (i);
+	  return -1;				/* errno is set for us.  */
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    return ctf_set_errno (fp, err);
+
+  return 0;
+}
+
+/* Look up a type kind from the output mapping, given a type hash value.  */
+static int
+ctf_dedup_hash_kind (ctf_file_t *fp, ctf_file_t **inputs, const char *hash)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  void *id;
+  ctf_dynset_t *type_ids;
+
+  /* Precondition: the output mapping is populated.  */
+  if (!ctf_assert (fp, ctf_dynhash_elements (d->cd_output_mapping) > 0))
+    return -1;
+
+  /* Look up some GID from the output hash for this type.  (They are all
+     identical, so we can pick any).  Don't assert if someone calls this
+     function wrongly, but do assert if the output mapping knows about the hash,
+     but has nothing associated with it.  */
+
+  type_ids = ctf_dynhash_lookup (d->cd_output_mapping, hash);
+  if (!type_ids)
+    {
+      ctf_dprintf ("Looked up type kind by nonexistent hash %s.\n", hash);
+      return ctf_set_errno (fp, ECTF_INTERNAL);
+    }
+  id = ctf_dynset_lookup_any (type_ids);
+  if (!ctf_assert (fp, id))
+    return -1;
+
+  return ctf_type_kind_unsliced (inputs[CTF_DEDUP_GID_TO_INPUT (id)],
+				 CTF_DEDUP_GID_TO_TYPE (id));
+}
+
+/* Used to keep a count of types: i.e. distinct type hash values.  */
+typedef struct ctf_dedup_type_counter
+{
+  ctf_file_t *fp;
+  ctf_file_t **inputs;
+  int num_non_forwards;
+} ctf_dedup_type_counter_t;
+
+/* Add to the type counter for one name entry from the cd_name_counts.  */
+static int
+ctf_dedup_count_types (void *key_, void *value _libctf_unused_, void *arg_)
+{
+  const char *hval = (const char *) key_;
+  int kind;
+  ctf_dedup_type_counter_t *arg = (ctf_dedup_type_counter_t *) arg_;
+
+  kind = ctf_dedup_hash_kind (arg->fp, arg->inputs, hval);
+
+  /* We rely on ctf_dedup_hash_kind setting the fp to -ECTF_INTERNAL on error to
+     smuggle errors out of here.  */
+
+  if (kind != CTF_K_FORWARD)
+    {
+      arg->num_non_forwards++;
+      ctf_dprintf ("Counting hash %s: kind %i: num_non_forwards is %i\n",
+		   hval, kind, arg->num_non_forwards);
+    }
+
+  /* We only need to know if there is more than one non-forward (an ambiguous
+     type): don't waste time iterating any more than needed to figure that
+     out.  */
+
+  if (arg->num_non_forwards > 1)
+    return 1;
+
+  return 0;
+}
+
+/* Detect name ambiguity and mark ambiguous names as conflicting, other than the
+   most common.  */
+static int
+ctf_dedup_detect_name_ambiguity (ctf_file_t *fp, ctf_file_t **inputs)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  ctf_next_t *i = NULL;
+  void *k;
+  void *v;
+  int err;
+  const char *erm;
+
+  /* Go through cd_name_counts for all CTF namespaces in turn.  */
+
+  while ((err = ctf_dynhash_next (d->cd_name_counts, &i, &k, &v)) == 0)
+    {
+      const char *decorated = (const char *) k;
+      ctf_dynhash_t *name_counts = (ctf_dynhash_t *) v;
+      ctf_next_t *j = NULL;
+
+      /* If this is a forwardable kind or a forward (which we can tell without
+	 consulting the type because its decorated name has a space as its
+	 second character: see ctf_decorate_type_name), we are only interested
+	 in whether this name has many hashes associated with it: any such name
+	 is necessarily ambiguous, and types with that name are conflicting.
+	 Once we know whether this is true, we can skip to the next name: so use
+	 ctf_dynhash_iter_find for efficiency.  */
+
+      if (decorated[0] != '\0' && decorated[1] == ' ')
+	{
+	  ctf_dedup_type_counter_t counters = { fp, inputs, 0 };
+	  ctf_dynhash_t *counts = (ctf_dynhash_t *) v;
+
+	  ctf_dynhash_iter_find (counts, ctf_dedup_count_types, &counters);
+
+	  /* Check for assertion failure and pass it up.  */
+	  if (ctf_errno (fp) == ECTF_INTERNAL)
+	    goto assert_err;
+
+	  if (counters.num_non_forwards > 1)
+	    {
+	      const void *hval_;
+
+	      while ((err = ctf_dynhash_cnext (counts, &j, &hval_, NULL)) == 0)
+		{
+		  const char *hval = (const char *) hval_;
+		  ctf_dynset_t *type_ids;
+		  void *id;
+		  int kind;
+
+		  /* Dig through the types in this hash to find the non-forwards
+		     and mark them ambiguous.  */
+
+		  type_ids = ctf_dynhash_lookup (d->cd_output_mapping, hval);
+
+		  /* Nonexistent? Must be a forward with no referent.  */
+		  if (!type_ids)
+		    continue;
+
+		  id = ctf_dynset_lookup_any (type_ids);
+
+		  kind = ctf_type_kind (inputs[CTF_DEDUP_GID_TO_INPUT (id)],
+					CTF_DEDUP_GID_TO_TYPE (id));
+
+		  if (kind != CTF_K_FORWARD)
+		    {
+		      ctf_dprintf ("Marking %p, with hash %s, conflicting: one "
+				   "of many non-forward GIDs for %s\n", id,
+				   hval, (char *) k);
+		      ctf_dedup_mark_conflicting_hash (fp, hval);
+		    }
+		}
+	      if (err != ECTF_NEXT_END)
+		{
+		  erm = "marking conflicting structs/unions";
+		  goto iterr;
+		}
+	    }
+	}
+      else
+	{
+	  /* This is an ordinary type.  Find the most common type with this
+	     name, and mark it unconflicting: all others are conflicting.  (We
+	     cannot do this sort of popularity contest with forwardable types
+	     because any forwards to that type would be immediately unified with
+	     the most-popular type on insertion, and we want conflicting structs
+	     et al to have all forwards left intact, so the user is notified
+	     that this type is conflicting.  TODO: improve this in future by
+	     setting such forwards non-root-visible.)  */
+
+	  const void *key;
+	  const void *count;
+	  const char *hval;
+	  long max_hcount = -1;
+	  const char *max_hval = NULL;
+
+	  if (ctf_dynhash_elements (name_counts) <= 1)
+	    continue;
+
+	  /* First find the most common.  */
+	  while ((err = ctf_dynhash_cnext (name_counts, &j, &key, &count)) == 0)
+	    {
+	      hval = (const char *) key;
+	      if ((long int) count > max_hcount)
+		{
+		  max_hcount = (long int) count;
+		  max_hval = hval;
+		}
+	    }
+	  if (err != ECTF_NEXT_END)
+	    {
+	      erm = "finding commonest conflicting type";
+	      goto iterr;
+	    }
+
+	  /* Mark all the others as conflicting.   */
+	  while ((err = ctf_dynhash_cnext (name_counts, &j, &key, NULL)) == 0)
+	    {
+	      hval = (const char *) key;
+	      if (strcmp (max_hval, hval) == 0)
+		continue;
+
+	      ctf_dprintf ("Marking %s, an uncommon hash for %s, conflicting\n",
+			   hval, (const char *) k);
+	      if (ctf_dedup_mark_conflicting_hash (fp, hval) < 0)
+		{
+		  erm = "marking hashes as conflicting";
+		  goto err;
+		}
+	    }
+	  if (err != ECTF_NEXT_END)
+	    {
+	      erm = "marking uncommon conflicting types";
+	      goto iterr;
+	    }
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    {
+      erm = "scanning for ambiguous names";
+      goto iterr;
+    }
+
+  return 0;
+
+ err:
+  ctf_next_destroy (i);
+  ctf_err_warn (fp, 0, "%s: %s", erm, ctf_errmsg (ctf_errno (fp)));
+  return -1;
+
+ iterr:
+  ctf_err_warn (fp, 0, "iteration failed %s: %s", erm, ctf_errmsg (err));
+  return ctf_set_errno (fp, err);
+
+ assert_err:
+  ctf_next_destroy (i);
+  return -1; 					/* errno is set for us.  */
+}
+
+/* Initialize the deduplication machinery.  */
+
+static int
+ctf_dedup_init (ctf_file_t *fp)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  size_t i;
+
+  if (ctf_dedup_atoms_init (fp) < 0)
+      goto oom;
+
+#if IDS_NEED_ALLOCATION
+  if ((d->cd_id_to_file_t = ctf_dynhash_create (ctf_hash_type_id_key,
+						ctf_hash_eq_type_id_key,
+						free, NULL)) == NULL)
+    goto oom;
+#endif
+
+  for (i = 0; i < 4; i++)
+    {
+      if ((d->cd_decorated_names[i] = ctf_dynhash_create (ctf_hash_string,
+							  ctf_hash_eq_string,
+							  NULL, NULL)) == NULL)
+	goto oom;
+    }
+
+  if ((d->cd_name_counts
+       = ctf_dynhash_create (ctf_hash_string,
+			     ctf_hash_eq_string, NULL,
+			     (ctf_hash_free_fun) ctf_dynhash_destroy)) == NULL)
+    goto oom;
+
+  if ((d->cd_type_hashes
+       = ctf_dynhash_create (ctf_hash_integer,
+			     ctf_hash_eq_integer,
+			     NULL, NULL)) == NULL)
+    goto oom;
+
+  if ((d->cd_struct_origin
+       = ctf_dynhash_create (ctf_hash_string,
+			     ctf_hash_eq_string,
+			     NULL, NULL)) == NULL)
+    goto oom;
+
+  if ((d->cd_citers
+       = ctf_dynhash_create (ctf_hash_string,
+			     ctf_hash_eq_string, NULL,
+			     (ctf_hash_free_fun) ctf_dynset_destroy)) == NULL)
+    goto oom;
+
+  if ((d->cd_output_mapping
+       = ctf_dynhash_create (ctf_hash_string,
+			     ctf_hash_eq_string, NULL,
+			     (ctf_hash_free_fun) ctf_dynset_destroy)) == NULL)
+    goto oom;
+
+  if ((d->cd_output_first_gid
+       = ctf_dynhash_create (ctf_hash_string,
+			     ctf_hash_eq_string,
+			     NULL, NULL)) == NULL)
+    goto oom;
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  if ((d->cd_output_mapping_guard
+       = ctf_dynhash_create (ctf_hash_integer,
+			     ctf_hash_eq_integer, NULL, NULL)) == NULL)
+    goto oom;
+#endif
+
+  if ((d->cd_emission_struct_members
+       = ctf_dynhash_create (ctf_hash_integer,
+			     ctf_hash_eq_integer,
+			     NULL, NULL)) == NULL)
+    goto oom;
+
+  if ((d->cd_conflicting_types
+       = ctf_dynset_create (htab_hash_string,
+			    ctf_dynset_eq_string, NULL)) == NULL)
+    goto oom;
+
+  return 0;
+
+ oom:
+  ctf_err_warn (fp, 0, "ctf_dedup_init: cannot initialize: "
+		"out of memory.");
+  return ctf_set_errno (fp, ENOMEM);
+}
+
+void
+ctf_dedup_fini (ctf_file_t *fp, ctf_file_t **outputs, uint32_t noutputs)
+{
+  ctf_dedup_t *d = &fp->ctf_dedup;
+  size_t i;
+
+  /* ctf_dedup_atoms is kept across links.  */
+#if IDS_NEED_ALLOCATION
+  ctf_dynhash_destroy (d->cd_id_to_file_t);
+#endif
+  for (i = 0; i < 4; i++)
+    ctf_dynhash_destroy (d->cd_decorated_names[i]);
+  ctf_dynhash_destroy (d->cd_name_counts);
+  ctf_dynhash_destroy (d->cd_type_hashes);
+  ctf_dynhash_destroy (d->cd_struct_origin);
+  ctf_dynhash_destroy (d->cd_citers);
+  ctf_dynhash_destroy (d->cd_output_mapping);
+  ctf_dynhash_destroy (d->cd_output_first_gid);
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+  ctf_dynhash_destroy (d->cd_output_mapping_guard);
+#endif
+  ctf_dynhash_destroy (d->cd_emission_struct_members);
+  ctf_dynset_destroy (d->cd_conflicting_types);
+
+  /* Free the per-output state.  */
+  if (outputs)
+    {
+      for (i = 0; i < noutputs; i++)
+	{
+	  ctf_dedup_t *od = &outputs[i]->ctf_dedup;
+	  ctf_dynhash_destroy (od->cd_output_emission_hashes);
+	  ctf_dynhash_destroy (od->cd_output_emission_conflicted_forwards);
+	  ctf_file_close (od->cd_output);
+	}
+    }
+  memset (d, 0, sizeof (ctf_dedup_t));
+}
+
+/* Return 1 if this type is cited by multiple input dictionaries.  */
+
+static int
+ctf_dedup_multiple_input_dicts (ctf_file_t *output, ctf_file_t **inputs,
+				const char *hval)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  ctf_dynset_t *type_ids;
+  ctf_next_t *i = NULL;
+  void *id;
+  ctf_file_t *found = NULL, *relative_found = NULL;
+  const char *type_id;
+  ctf_file_t *input_fp;
+  ctf_id_t input_id;
+  const char *name;
+  const char *decorated;
+  int fwdkind;
+  int multiple = 0;
+  int err;
+
+  type_ids = ctf_dynhash_lookup (d->cd_output_mapping, hval);
+  if (!ctf_assert (output, type_ids))
+    return -1;
+
+  /* Scan across the IDs until we find proof that two disjoint dictionaries
+     are referenced.  Exit as soon as possible.  Optimization opportunity, but
+     possibly not worth it, given that this is only executed in
+     CTF_LINK_SHARE_DUPLICATED mode.  */
+
+  while ((err = ctf_dynset_next (type_ids, &i, &id)) == 0)
+    {
+      ctf_file_t *fp = inputs[CTF_DEDUP_GID_TO_INPUT (id)];
+
+      if (fp == found || fp == relative_found)
+	continue;
+
+      if (!found)
+	{
+	  found = fp;
+	  continue;
+	}
+
+      if (!relative_found
+	  && (fp->ctf_parent == found || found->ctf_parent == fp))
+	{
+	  relative_found = fp;
+	  continue;
+	}
+
+      multiple = 1;
+      ctf_next_destroy (i);
+      break;
+    }
+  if ((err != ECTF_NEXT_END) && (err != 0))
+    {
+      ctf_err_warn (output, 0, "propagating conflictedness: %s",
+		    ctf_errmsg (err));
+      ctf_set_errno (output, err);
+      return -1;
+    }
+
+  if (multiple)
+    return multiple;
+
+  /* This type itself does not appear in multiple input dicts: how about another
+     related type with the same name (e.g. a forward if this is a struct,
+     etc).  */
+
+  type_id = ctf_dynset_lookup_any (type_ids);
+  if (!ctf_assert (output, type_id))
+    return -1;
+
+  input_fp = inputs[CTF_DEDUP_GID_TO_INPUT (type_id)];
+  input_id = CTF_DEDUP_GID_TO_TYPE (type_id);
+  fwdkind = ctf_type_kind_forwarded (input_fp, input_id);
+  name = ctf_type_name_raw (input_fp, input_id);
+
+  if ((fwdkind == CTF_K_STRUCT || fwdkind == CTF_K_UNION)
+      && name && name[0] != '\0')
+    {
+      const void *origin;
+
+      if ((decorated = ctf_decorate_type_name (output, name,
+					       fwdkind)) == NULL)
+	return -1;				/* errno is set for us.  */
+
+      origin = ctf_dynhash_lookup (d->cd_struct_origin, decorated);
+      if ((origin != NULL) && (CTF_DEDUP_GID_TO_INPUT (origin) < 0))
+	multiple = 1;
+    }
+
+  return multiple;
+}
+
+/* Demote unconflicting types which reference only one input, or which reference
+   two inputs where one input is the parent of the other, into conflicting
+   types.  Only used if the link mode is CTF_LINK_SHARE_DUPLICATED.  */
+
+static int
+ctf_dedup_conflictify_unshared (ctf_file_t *output, ctf_file_t **inputs)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  ctf_next_t *i = NULL;
+  int err;
+  const void *k;
+  ctf_dynset_t *to_mark = NULL;
+
+  if ((to_mark = ctf_dynset_create (htab_hash_string, ctf_dynset_eq_string,
+				    NULL)) == NULL)
+    goto err_no;
+
+  while ((err = ctf_dynhash_cnext (d->cd_output_mapping, &i, &k, NULL)) == 0)
+    {
+      const char *hval = (const char *) k;
+      int conflicting;
+
+      /* Types referenced by only one dict, with no type appearing under that
+	 name elsewhere, are marked conflicting.  */
+
+      conflicting = !ctf_dedup_multiple_input_dicts (output, inputs, hval);
+
+      if (conflicting < 0)
+	goto err;				/* errno is set for us.  */
+
+      if (conflicting)
+	if (ctf_dynset_cinsert (to_mark, hval) < 0)
+	  goto err;
+    }
+  if (err != ECTF_NEXT_END)
+    goto iterr;
+
+  while ((err = ctf_dynset_cnext (to_mark, &i, &k)) == 0)
+    {
+      const char *hval = (const char *) k;
+
+      if (ctf_dedup_mark_conflicting_hash (output, hval) < 0)
+	goto err;
+    }
+  if (err != ECTF_NEXT_END)
+    goto iterr;
+
+  ctf_dynset_destroy (to_mark);
+
+  return 0;
+
+ err_no:
+  ctf_set_errno (output, errno);
+ err:
+  err = ctf_errno (output);
+  ctf_next_destroy (i);
+ iterr:
+  ctf_set_errno (output, err);
+  ctf_dynset_destroy (to_mark);
+  ctf_err_warn (output, 0, "conflictifying unshared types: %s",
+		ctf_errmsg (ctf_errno (output)));
+  return -1;
+}
+
+/* The core deduplicator.  Populate cd_output_mapping in the output ctf_dedup
+   with a mapping of all types that belong in this dictionary and where they
+   come from, and cd_conflicting_types with an indication of whether each type
+   is conflicted or not.  OUTPUT is the top-level output: INPUTS is the array of
+   input dicts; NINPUTS is the size of that array; PARENTS is an NINPUTS-element
+   array with each element corresponding to a input which is a child dict set to
+   the number in the INPUTS array of that input's parent.
+
+   If CU_MAPPED is set, this is a first pass for a link with a non-empty CU
+   mapping: only one output will result.
+
+   Only deduplicates: does not emit the types into the output.  Call
+   ctf_dedup_emit afterwards to do that.  */
+
+int
+ctf_dedup (ctf_file_t *output, ctf_file_t **inputs, uint32_t ninputs,
+	   uint32_t *parents, int cu_mapped)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  size_t i;
+  ctf_next_t *it = NULL;
+
+  for (i = 0; i < ninputs; i++)
+    ctf_dprintf ("Input %zi: %s\n", i, ctf_link_input_name (inputs[i]));
+
+  if (ctf_dedup_init (output) < 0)
+    return -1; 					/* errno is set for us.  */
+
+  /* Some flags do not apply when CU-mapping: this is not a duplicated link,
+     because there is only one output and we really don't want to end up marking
+     all nonconflicting but appears-only-once types as conflicting (which in the
+     CU-mapped link means we'd mark them all as non-root-visible!).  */
+  d->cd_link_flags = output->ctf_link_flags;
+  if (cu_mapped)
+    d->cd_link_flags &= ~(CTF_LINK_SHARE_DUPLICATED);
+
+  /* Compute hash values for all types, recursively, treating child structures
+     and unions equivalent to forwards, and hashing in the name of the referent
+     of each such type into structures, unions, and non-opaque forwards.
+     Populate a mapping from decorated name (including an indication of
+     struct/union/enum namespace) to count of type hash values in
+     cd_name_counts, a mapping from and a mapping from hash values to input type
+     IDs in cd_output_mapping.  */
+
+  ctf_dprintf ("Computing type hashes\n");
+  for (i = 0; i < ninputs; i++)
+    {
+      ctf_id_t id;
+
+      while ((id = ctf_type_next (inputs[i], &it, NULL, 1)) != CTF_ERR)
+	{
+	  ctf_dedup_hash_type (output, inputs[i], inputs, parents,
+			       i, id, 0, 0, ctf_dedup_populate_mappings);
+	}
+      if (ctf_errno (inputs[i]) != ECTF_NEXT_END)
+	{
+	  ctf_err_warn (output, 0, "iteration failure computing type "
+			"hashes: %s", ctf_errmsg (ctf_errno (inputs[i])));
+	  return ctf_set_errno (output, ctf_errno (inputs[i]));
+	}
+    }
+
+  /* Go through the cd_name_counts name->hash->count mapping for all CTF
+     namespaces: any name with many hashes associated with it at this stage is
+     necessarily ambiguous.  Mark all the hashes except the most common as
+     conflicting in the output.  */
+
+  ctf_dprintf ("Detecting type name ambiguity\n");
+  if (ctf_dedup_detect_name_ambiguity (output, inputs) < 0)
+    return -1;					/* errno is set for us.  */
+
+  /* If the link mode is CTF_LINK_SHARE_DUPLICATED, we change any unconflicting
+     types whose output mapping references only one input dict into a
+     conflicting type, so that they end up in the per-CU dictionaries.  */
+
+  if (d->cd_link_flags & CTF_LINK_SHARE_DUPLICATED)
+    {
+      ctf_dprintf ("Conflictifying unshared types\n");
+      if (ctf_dedup_conflictify_unshared (output, inputs) < 0)
+	return -1;				/* errno is set for us.  */
+    }
+  return 0;
+}
+
+static int
+ctf_dedup_rwalk_output_mapping (ctf_file_t *output, ctf_file_t **inputs,
+				uint32_t ninputs, uint32_t *parents,
+				ctf_dynset_t *already_visited,
+				const char *hval,
+				int (*visit_fun) (const char *hval,
+						  ctf_file_t *output,
+						  ctf_file_t **inputs,
+						  uint32_t ninputs,
+						  uint32_t *parents,
+						  int already_visited,
+						  ctf_file_t *input,
+						  ctf_id_t type,
+						  void *id,
+						  int depth,
+						  void *arg),
+				void *arg, unsigned long depth);
+
+/* Like ctf_dedup_rwalk_output_mapping (which see), only takes a single target
+   type and visits it.  */
+static int
+ctf_dedup_rwalk_one_output_mapping (ctf_file_t *output,
+				    ctf_file_t **inputs, uint32_t ninputs,
+				    uint32_t *parents,
+				    ctf_dynset_t *already_visited,
+				    int visited, void *type_id,
+				    const char *hval,
+				    int (*visit_fun) (const char *hval,
+						      ctf_file_t *output,
+						      ctf_file_t **inputs,
+						      uint32_t ninputs,
+						      uint32_t *parents,
+						      int already_visited,
+						      ctf_file_t *input,
+						      ctf_id_t type,
+						      void *id,
+						      int depth,
+						      void *arg),
+				    void *arg, unsigned long depth)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  ctf_file_t *fp;
+  int input_num;
+  ctf_id_t type;
+  int ret;
+  const char *whaterr;
+
+  input_num = CTF_DEDUP_GID_TO_INPUT (type_id);
+  fp = inputs[input_num];
+  type = CTF_DEDUP_GID_TO_TYPE (type_id);
+
+  ctf_dprintf ("%lu: Starting walk over type %s, %i/%lx (%p), from %s, "
+	       "kind %i\n", depth, hval, input_num, type, (void *) fp,
+	       ctf_link_input_name (fp), ctf_type_kind_unsliced (fp, type));
+
+  /* Get the single call we do if this type has already been visited out of the
+     way.  */
+  if (visited)
+    return visit_fun (hval, output, inputs, ninputs, parents, visited, fp,
+		      type, type_id, depth, arg);
+
+  /* This macro is really ugly, but the alternative is repeating this code many
+     times, which is worse.  */
+
+#define CTF_TYPE_WALK(type, errlabel, errmsg)				\
+  do {									\
+    void *type_id;							\
+    const char *hashval;						\
+    int cited_type_input_num = input_num;				\
+									\
+    if ((fp->ctf_flags & LCTF_CHILD) && (LCTF_TYPE_ISPARENT (fp, type))) \
+      cited_type_input_num = parents[input_num];			\
+									\
+    type_id = CTF_DEDUP_GID (output, cited_type_input_num, type);	\
+									\
+    if (type == 0)							\
+      {									\
+	ctf_dprintf ("Walking: unimplemented type\n");			\
+	break;								\
+      }									\
+									\
+    ctf_dprintf ("Looking up ID %i/%lx in type hashes\n",		\
+		 cited_type_input_num, type);				\
+    hashval = ctf_dynhash_lookup (d->cd_type_hashes, type_id);		\
+    if (!ctf_assert (output, hashval))					\
+      {									\
+	whaterr = "looking up ID in type hashes";			\
+	goto errlabel;							\
+      }									\
+    ctf_dprintf ("ID %i/%lx has hash %s\n", cited_type_input_num, type,	\
+		 hashval);						\
+									\
+    ret = ctf_dedup_rwalk_output_mapping (output, inputs, ninputs, parents, \
+					  already_visited, hashval,	\
+					  visit_fun, arg, depth);	\
+    if (ret < 0)							\
+      {									\
+	whaterr = errmsg;						\
+	goto errlabel;							\
+      }									\
+  } while (0)
+
+  switch (ctf_type_kind_unsliced (fp, type))
+    {
+    case CTF_K_UNKNOWN:
+      /* Just skip things of unknown kind.  */
+      return 0;
+    case CTF_K_FORWARD:
+    case CTF_K_INTEGER:
+    case CTF_K_FLOAT:
+    case CTF_K_ENUM:
+      /* No types referenced.  */
+      break;
+
+    case CTF_K_TYPEDEF:
+    case CTF_K_VOLATILE:
+    case CTF_K_CONST:
+    case CTF_K_RESTRICT:
+    case CTF_K_POINTER:
+    case CTF_K_SLICE:
+      CTF_TYPE_WALK (ctf_type_reference (fp, type), err,
+		     "Referenced type walk");
+      break;
+
+    case CTF_K_ARRAY:
+      {
+	ctf_arinfo_t ar;
+
+	if (ctf_array_info (fp, type, &ar) < 0)
+	  {
+	    whaterr = "array info lookup";
+	    goto err_msg;
+	  }
+
+	CTF_TYPE_WALK (ar.ctr_contents, err, "Array contents type walk");
+	CTF_TYPE_WALK (ar.ctr_index, err, "Array index type walk");
+	break;
+      }
+
+    case CTF_K_FUNCTION:
+      {
+	ctf_funcinfo_t fi;
+	ctf_id_t *args;
+	uint32_t j;
+
+	if (ctf_func_type_info (fp, type, &fi) < 0)
+	  {
+	    whaterr = "func type info lookup";
+	    goto err_msg;
+	  }
+
+	CTF_TYPE_WALK (fi.ctc_return, err, "Func return type walk");
+
+	if ((args = calloc (fi.ctc_argc, sizeof (ctf_id_t))) == NULL)
+	  {
+	    whaterr = "memory allocation";
+	    goto err_msg;
+	  }
+
+	if (ctf_func_type_args (fp, type, fi.ctc_argc, args) < 0)
+	  {
+	    whaterr = "func arg type lookup";
+	    free (args);
+	    goto err_msg;
+	  }
+
+	for (j = 0; j < fi.ctc_argc; j++)
+	  CTF_TYPE_WALK (args[j], err_free_args, "Func arg type walk");
+	free (args);
+	break;
+
+      err_free_args:
+	free (args);
+	goto err;
+      }
+    case CTF_K_STRUCT:
+    case CTF_K_UNION:
+      /* We do not recursively traverse the members of structures: they are
+	 emitted later, in a separate pass.  */
+	break;
+    default:
+      whaterr = "unknown type kind";
+      goto err;
+    }
+
+  return visit_fun (hval, output, inputs, ninputs, parents, visited, fp, type,
+		    type_id, depth, arg);
+
+ err_msg:
+  ctf_set_errno (output, ctf_errno (fp));
+  ctf_err_warn (fp, 0, "%s during type walking in %s at ID %lx: %s", whaterr,
+	       ctf_link_input_name (fp), type, ctf_errmsg (ctf_errno (fp)));
+ err:
+  return -1;
+}
+/* Recursively traverse the output mapping, and do something with each type
+   visited, from leaves to root.  VISIT_FUN, called as recursion unwinds,
+   returns a negative error code or zero.  Type hashes may be visited more than
+   once, but are not recursed through repeatedly: ALREADY_VISITED tracks whether
+   types have already been visited.  */
+static int
+ctf_dedup_rwalk_output_mapping (ctf_file_t *output, ctf_file_t **inputs,
+				uint32_t ninputs, uint32_t *parents,
+				ctf_dynset_t *already_visited,
+				const char *hval,
+				int (*visit_fun) (const char *hval,
+						  ctf_file_t *output,
+						  ctf_file_t **inputs,
+						  uint32_t ninputs,
+						  uint32_t *parents,
+						  int already_visited,
+						  ctf_file_t *input,
+						  ctf_id_t type,
+						  void *id,
+						  int depth,
+						  void *arg),
+				void *arg, unsigned long depth)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  ctf_next_t *i = NULL;
+  int err;
+  int visited = 1;
+  ctf_dynset_t *type_ids;
+  void *id;
+
+  depth++;
+
+  type_ids = ctf_dynhash_lookup (d->cd_output_mapping, hval);
+  if (!type_ids)
+    {
+      ctf_err_warn (output, 0, "looked up type kind by nonexistent "
+		    "hash %s.", hval);
+      return ctf_set_errno (output, ECTF_INTERNAL);
+    }
+
+  /* Have we seen this type before?  */
+
+  if (!ctf_dynset_exists (already_visited, hval, NULL))
+    {
+      /* Mark as already-visited immediately, to eliminate the possibility of
+	 cycles: but remember we have not actually visited it yet for the
+	 upcoming call to the visit_fun.  (All our callers handle cycles
+	 properly themselves, so we can just abort them aggressively as soon as
+	 we find ourselves in one.)  */
+
+      visited = 0;
+      if (ctf_dynset_cinsert (already_visited, hval) < 0)
+	{
+	  ctf_err_warn (output, 0, "out of memory tracking already-visited "
+			"types.");
+	  return ctf_set_errno (output, ENOMEM);
+	}
+    }
+
+  /* If this type is marked conflicted, traverse members and call
+     ctf_dedup_rwalk_output_mapping_once on all the unique ones: otherwise, just
+     pick a random one and use it.  */
+
+  if (!ctf_dynset_exists (d->cd_conflicting_types, hval, NULL))
+    {
+      id = ctf_dynset_lookup_any (type_ids);
+      if (!ctf_assert (output, id))
+	return -1;
+
+      return ctf_dedup_rwalk_one_output_mapping (output, inputs, ninputs,
+						 parents, already_visited,
+						 visited, id, hval, visit_fun,
+						 arg, depth);
+    }
+
+  while ((err = ctf_dynset_next (type_ids, &i, &id)) == 0)
+    {
+      int ret;
+
+      ret = ctf_dedup_rwalk_one_output_mapping (output, inputs, ninputs,
+						parents, already_visited,
+						visited, id, hval,
+						visit_fun, arg, depth);
+      if (ret < 0)
+	{
+	  ctf_next_destroy (i);
+	  return ret;				/* errno is set for us.  */
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    {
+      ctf_err_warn (output, 0, "walking many types with one hash: %s",
+		    ctf_errmsg (err));
+      return ctf_set_errno (output, err);
+    }
+
+  return 0;
+}
+
+typedef struct ctf_sort_om_cb_arg
+{
+  ctf_file_t **inputs;
+  uint32_t ninputs;
+  ctf_dedup_t *d;
+} ctf_sort_om_cb_arg_t;
+
+/* Sort the output mapping into order: types first appearing in earlier inputs
+   first, parents preceding children: if types first appear in the same input,
+   sort those with earlier ctf_id_t's first.  */
+static int
+sort_output_mapping (const ctf_next_hkv_t *one, const ctf_next_hkv_t *two,
+		     void *arg_)
+{
+  ctf_sort_om_cb_arg_t *arg = (ctf_sort_om_cb_arg_t *) arg_;
+  ctf_dedup_t *d = arg->d;
+  const char *one_hval = (const char *) one->hkv_key;
+  const char *two_hval = (const char *) two->hkv_key;
+  void *one_gid, *two_gid;
+  uint32_t one_ninput;
+  uint32_t two_ninput;
+  ctf_file_t *one_fp;
+  ctf_file_t *two_fp;
+  ctf_id_t one_type;
+  ctf_id_t two_type;
+
+  one_gid = ctf_dynhash_lookup (d->cd_output_first_gid, one_hval);
+  two_gid = ctf_dynhash_lookup (d->cd_output_first_gid, two_hval);
+
+  one_ninput = CTF_DEDUP_GID_TO_INPUT (one_gid);
+  two_ninput = CTF_DEDUP_GID_TO_INPUT (two_gid);
+
+  one_type = CTF_DEDUP_GID_TO_TYPE (one_gid);
+  two_type = CTF_DEDUP_GID_TO_TYPE (two_gid);
+
+  /* It's kind of hard to smuggle an assertion failure out of here.  */
+  assert (one_ninput < arg->ninputs && two_ninput < arg->ninputs);
+
+  one_fp = arg->inputs[one_ninput];
+  two_fp = arg->inputs[two_ninput];
+
+  /* Parents before children.  */
+
+  if (!(one_fp->ctf_flags & LCTF_CHILD)
+      && (two_fp->ctf_flags & LCTF_CHILD))
+    return -1;
+  else if ((one_fp->ctf_flags & LCTF_CHILD)
+      && !(two_fp->ctf_flags & LCTF_CHILD))
+    return 1;
+
+  /* ninput order, types appearing in earlier TUs first.  */
+
+  if (one_ninput < two_ninput)
+    return -1;
+  else if (two_ninput < one_ninput)
+    return 1;
+
+  /* Same TU.  Earliest ctf_id_t first.  They cannot be the same.  */
+
+  assert (one_type != two_type);
+  if (one_type < two_type)
+    return -1;
+  else
+    return 1;
+}
+
+/* The public entry point to ctf_dedup_rwalk_output_mapping, above.  */
+static int
+ctf_dedup_walk_output_mapping (ctf_file_t *output, ctf_file_t **inputs,
+			       uint32_t ninputs, uint32_t *parents,
+			       int (*visit_fun) (const char *hval,
+						 ctf_file_t *output,
+						 ctf_file_t **inputs,
+						 uint32_t ninputs,
+						 uint32_t *parents,
+						 int already_visited,
+						 ctf_file_t *input,
+						 ctf_id_t type,
+						 void *id,
+						 int depth,
+						 void *arg),
+			       void *arg)
+{
+  ctf_dynset_t *already_visited;
+  ctf_next_t *i = NULL;
+  ctf_sort_om_cb_arg_t sort_arg;
+  int err;
+  void *k;
+
+  if ((already_visited = ctf_dynset_create (htab_hash_string,
+					    ctf_dynset_eq_string,
+					    NULL)) == NULL)
+    return ctf_set_errno (output, ENOMEM);
+
+  sort_arg.inputs = inputs;
+  sort_arg.ninputs = ninputs;
+  sort_arg.d = &output->ctf_dedup;
+
+  while ((err = ctf_dynhash_next_sorted (output->ctf_dedup.cd_output_mapping,
+					 &i, &k, NULL, sort_output_mapping,
+					 &sort_arg)) == 0)
+    {
+      const char *hval = (const char *) k;
+
+      err = ctf_dedup_rwalk_output_mapping (output, inputs, ninputs, parents,
+					    already_visited, hval, visit_fun,
+					    arg, 0);
+      if (err < 0)
+	{
+	  ctf_next_destroy (i);
+	  goto err;				/* errno is set for us.  */
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    {
+      ctf_err_warn (output, 0, "recursing over output mapping: %s",
+		    ctf_errmsg (err));
+      ctf_set_errno (output, err);
+      goto err;
+    }
+  ctf_dynset_destroy (already_visited);
+
+  return 0;
+ err:
+  ctf_dynset_destroy (already_visited);
+  return -1;
+}
+
+/* Possibly synthesise a synthetic forward in TARGET to subsitute for a
+   conflicted per-TU type ID in INPUT with hash HVAL.  Return its CTF ID, or 0
+   if none was needed.  */
+static ctf_id_t
+ctf_dedup_maybe_synthesize_forward (ctf_file_t *output, ctf_file_t *target,
+				    ctf_file_t *input, ctf_id_t id,
+				    const char *hval)
+{
+  ctf_dedup_t *od = &output->ctf_dedup;
+  ctf_dedup_t *td = &target->ctf_dedup;
+  int kind;
+  int fwdkind;
+  const char *name;
+  const char *decorated;
+  void *v;
+  ctf_id_t emitted_forward;
+
+  if (!ctf_dynset_exists (od->cd_conflicting_types, hval, NULL)
+      || target->ctf_flags & LCTF_CHILD
+      || !ctf_type_name_raw (input, id)
+      || (((kind = ctf_type_kind_unsliced (input, id)) != CTF_K_STRUCT
+	   && kind != CTF_K_UNION && kind != CTF_K_FORWARD)))
+    return 0;
+
+  fwdkind = ctf_type_kind_forwarded (input, id);
+  name = ctf_type_name_raw (input, id);
+
+  ctf_dprintf ("Using synthetic forward for conflicted struct/union with "
+	       "hval %s\n", hval);
+
+  if (!ctf_assert (output, name))
+    return CTF_ERR;
+
+  if ((decorated = ctf_decorate_type_name (output, name, fwdkind)) == NULL)
+    return CTF_ERR;
+
+  if (!ctf_dynhash_lookup_kv (td->cd_output_emission_conflicted_forwards,
+			      decorated, NULL, &v))
+    {
+      if ((emitted_forward = ctf_add_forward (target, CTF_ADD_ROOT, name,
+					      fwdkind)) == CTF_ERR)
+	{
+	  ctf_set_errno (output, ctf_errno (target));
+	  return CTF_ERR;
+	}
+
+      if (ctf_dynhash_cinsert (td->cd_output_emission_conflicted_forwards,
+			       decorated, (void *) emitted_forward) < 0)
+	{
+	  ctf_set_errno (output, ENOMEM);
+	  return CTF_ERR;
+	}
+    }
+  else
+    emitted_forward = (ctf_id_t) (uintptr_t) v;
+
+  ctf_dprintf ("Cross-TU conflicted struct: passing back forward, %lx\n",
+	       emitted_forward);
+
+  return emitted_forward;
+}
+
+/* Map a GID in some INPUT dict, in the form of an input number and a ctf_id_t,
+   into a GID in a target output dict.  If it returns 0, this is the
+   unimplemented type, and the input type must have been 0.  The OUTPUT dict is
+   assumed to be the parent of the TARGET, if it is not the TARGET itself.
+
+   Returns CTF_ERR on failure.  Responds to an incoming CTF_ERR as an 'id' by
+   returning CTF_ERR, to simplify callers.  Errors are always propagated to the
+   input, even if they relate to the target, for the same reason.  (Target
+   errors are expected to be very rare.)
+
+   If the type in question is a citation of a conflicted type in a different TU,
+   emit a forward of the right type in its place (if not already emitted), and
+   record that forward in cd_output_emission_conflicted_forwards.  This avoids
+   the need to replicate the entire type graph below this point in the current
+   TU (an appalling waste of space).
+
+   TODO: maybe replace forwards in the same TU with their referents?  Might
+   make usability a bit better.  */
+
+static ctf_id_t
+ctf_dedup_id_to_target (ctf_file_t *output, ctf_file_t *target,
+			ctf_file_t **inputs, uint32_t ninputs,
+			uint32_t *parents, ctf_file_t *input, int input_num,
+			ctf_id_t id)
+{
+  ctf_dedup_t *od = &output->ctf_dedup;
+  ctf_dedup_t *td = &target->ctf_dedup;
+  ctf_file_t *err_fp = input;
+  const char *hval;
+  void *target_id;
+  ctf_id_t emitted_forward;
+
+  /* The target type of an error is an error.  */
+  if (id == CTF_ERR)
+    return CTF_ERR;
+
+  /* The unimplemented type's ID never changes.  */
+  if (!id)
+    {
+      ctf_dprintf ("%i/%lx: unimplemented type\n", input_num, id);
+      return 0;
+    }
+
+  ctf_dprintf ("Mapping %i/%lx to target %p (%s)\n", input_num,
+	       id, (void *) target, ctf_link_input_name (target));
+
+  /* If the input type is in the parent type space, and this is a child, reset
+     the input to the parent (which must already have been emitted, since
+     emission of parent dicts happens before children).  */
+  if ((input->ctf_flags & LCTF_CHILD) && (LCTF_TYPE_ISPARENT (input, id)))
+    {
+      if (!ctf_assert (output, parents[input_num] <= ninputs))
+	return -1;
+      input = inputs[parents[input_num]];
+      input_num = parents[input_num];
+    }
+
+  hval = ctf_dynhash_lookup (od->cd_type_hashes,
+			     CTF_DEDUP_GID (output, input_num, id));
+
+  if (!ctf_assert (output, hval && td->cd_output_emission_hashes))
+    return -1;
+
+  /* If this type is a conflicted tagged structure, union, or forward,
+     substitute a synthetic forward instead, emitting it if need be.  Only do
+     this if the target is in the parent dict: if it's in the child dict, we can
+     just point straight at the thing itself.  Of course, we might be looking in
+     the child dict right now and not find it and have to look in the parent, so
+     we have to do this check twice.  */
+
+  emitted_forward = ctf_dedup_maybe_synthesize_forward (output, target,
+							input, id, hval);
+  switch (emitted_forward)
+    {
+    case 0: /* No forward needed.  */
+      break;
+    case -1:
+      ctf_set_errno (err_fp, ctf_errno (output));
+      ctf_err_warn (err_fp, 0, "adding synthetic forward for type %i/%lx: "
+		    "%s", input_num, id, ctf_errmsg (ctf_errno (err_fp)));
+      return -1;
+    default:
+      return emitted_forward;
+    }
+
+  ctf_dprintf ("Looking up %i/%lx, hash %s, in target\n", input_num, id, hval);
+
+  target_id = ctf_dynhash_lookup (td->cd_output_emission_hashes, hval);
+  if (!target_id)
+    {
+      /* Must be in the parent, so this must be a child, and they must not be
+	 the same dict.  */
+      ctf_dprintf ("Checking shared parent for target\n");
+      if (!ctf_assert (output, (target != output)
+		       && (target->ctf_flags & LCTF_CHILD)))
+	return -1;
+
+      target_id = ctf_dynhash_lookup (od->cd_output_emission_hashes, hval);
+
+      emitted_forward = ctf_dedup_maybe_synthesize_forward (output, output,
+							    input, id, hval);
+      switch (emitted_forward)
+	{
+	case 0: /* No forward needed.  */
+	  break;
+	case -1:
+	  ctf_set_errno (err_fp, ctf_errno (output));
+	  ctf_err_warn (err_fp, 0, "adding synthetic forward for type %i/%lx: "
+			"%s", input_num, id, ctf_errmsg (ctf_errno (err_fp)));
+	  return -1;
+	default:
+	  return emitted_forward;
+	}
+    }
+  if (!ctf_assert (output, target_id))
+    return -1;
+  return (ctf_id_t) target_id;
+}
+
+/* Emit a single deduplicated TYPE with the given HVAL, located in a given
+   INPUT, with the given (G)ID, into the shared OUTPUT or a
+   possibly-newly-created per-CU dict.  All the types this type depends upon
+   have already been emitted.  (This type itself may also have been emitted.)
+
+   If the ARG is 1, this is a CU-mapped deduplication round mapping many
+   ctf_file_t's into precisely one: conflicting types should be marked
+   non-root-visible.  If the ARG is 0, conflicting types go into per-CU
+   dictionaries stored in the input's ctf_dedup.cd_output: otherwise, everything
+   is emitted directly into the output.  No struct/union members are emitted.
+
+   Optimization opportunity: trace the ancestry of non-root-visible types and
+   elide all that neither have a root-visible type somewhere towards their root,
+   nor have the type visible via any other route (the function info section,
+   data object section, backtrace section etc).  */
+
+static int
+ctf_dedup_emit_type (const char *hval, ctf_file_t *output, ctf_file_t **inputs,
+		     uint32_t ninputs, uint32_t *parents, int already_visited,
+		     ctf_file_t *input, ctf_id_t type, void *id, int depth,
+		     void *arg)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  int kind = ctf_type_kind_unsliced (input, type);
+  const char *name;
+  ctf_file_t *target = output;
+  ctf_file_t *real_input;
+  const ctf_type_t *tp;
+  int input_num = CTF_DEDUP_GID_TO_INPUT (id);
+  int output_num = (uint32_t) -1;		/* 'shared' */
+  int cu_mapped = *(int *)arg;
+  int isroot = 1;
+  int is_conflicting;
+
+  ctf_next_t *i = NULL;
+  ctf_id_t new_type;
+  ctf_id_t ref;
+  ctf_id_t maybe_dup = 0;
+  ctf_encoding_t ep;
+  const char *erm;
+  int emission_hashed = 0;
+
+  /* We don't want to re-emit something we've already emitted.  */
+
+  if (already_visited)
+    return 0;
+
+  ctf_dprintf ("%i: Emitting type with hash %s from %s: determining target\n",
+	       depth, hval, ctf_link_input_name (input));
+
+  /* Conflicting types go into a per-CU output dictionary, unless this is a
+     CU-mapped run.  The import is not refcounted, since it goes into the
+     ctf_link_outputs dict of the output that is its parent.  */
+  is_conflicting = ctf_dynset_exists (d->cd_conflicting_types, hval, NULL);
+
+  if (is_conflicting && !cu_mapped)
+    {
+      ctf_dprintf ("%i: Type %s in %i/%lx is conflicted: "
+		   "inserting into per-CU target.\n",
+		   depth, hval, input_num, type);
+
+      if (input->ctf_dedup.cd_output)
+	target = input->ctf_dedup.cd_output;
+      else
+	{
+	  int err;
+
+	  if ((target = ctf_create (&err)) == NULL)
+	    {
+	      ctf_err_warn (output, 0, "cannot create per-CU CTF archive "
+			    "for CU %s: %s", ctf_link_input_name (input),
+			    ctf_errmsg (err)); ctf_set_errno (output, err);
+	      return -1;
+	    }
+
+	  ctf_import_unref (target, output);
+	  if (ctf_cuname (input) != NULL)
+	    ctf_cuname_set (target, ctf_cuname (input));
+	  else
+	    ctf_cuname_set (target, "unnamed-CU");
+	  ctf_parent_name_set (target, _CTF_SECTION);
+
+	  input->ctf_dedup.cd_output = target;
+	}
+      output_num = input_num;
+    }
+
+  real_input = input;
+  if ((tp = ctf_lookup_by_id (&real_input, type)) == NULL)
+    {
+      ctf_err_warn (output, 0, "%s: lookup failure for type %lx: %s",
+		    ctf_link_input_name (real_input), type,
+		    ctf_errmsg (ctf_errno (input)));
+      ctf_set_errno (output, ctf_errno (input));
+      return -1;		/* errno is set for us.  */
+    }
+
+  name = ctf_strraw (real_input, tp->ctt_name);
+
+  /* Hide conflicting types, if we were asked to: also hide if a type with this
+     name already exists and is not a forward.  */
+  if (cu_mapped && is_conflicting)
+    isroot = 0;
+  else if (name
+	   && (maybe_dup = ctf_lookup_by_rawname (target, kind, name)) != 0)
+    {
+      if (ctf_type_kind (target, maybe_dup) != CTF_K_FORWARD)
+	isroot = 0;
+    }
+
+  ctf_dprintf ("%i: Emitting type with hash %s (%s), into target %i/%p\n",
+	       depth, hval, name ? name : "", input_num, (void *) target);
+
+  if (!target->ctf_dedup.cd_output_emission_hashes)
+    if ((target->ctf_dedup.cd_output_emission_hashes
+	 = ctf_dynhash_create (ctf_hash_string, ctf_hash_eq_string,
+			      NULL, NULL)) == NULL)
+      goto oom_hash;
+
+  if (!target->ctf_dedup.cd_output_emission_conflicted_forwards)
+    if ((target->ctf_dedup.cd_output_emission_conflicted_forwards
+	 = ctf_dynhash_create (ctf_hash_string, ctf_hash_eq_string,
+			      NULL, NULL)) == NULL)
+      goto oom_hash;
+
+  switch (kind)
+    {
+    case CTF_K_UNKNOWN:
+      /* These are types that CTF cannot encode, marked as such by the compile.
+	 We intentionally do not re-emit these.  */
+      new_type = 0;
+      break;
+    case CTF_K_FORWARD:
+      /* This will do nothing if the type to which this forwards already exists,
+	 and will be replaced with such a type if it appears later.  */
+
+      erm = "forward";
+      if ((new_type = ctf_add_forward (target, isroot, name,
+				       ctf_type_kind_forwarded (input, type)))
+	  == CTF_ERR)
+	goto err_target;
+      break;
+
+    case CTF_K_FLOAT:
+    case CTF_K_INTEGER:
+      erm = "float/int";
+      if (ctf_type_encoding (input, type, &ep) < 0)
+	goto err_input;				/* errno is set for us.  */
+      if ((new_type = ctf_add_encoded (target, isroot, name, &ep, kind))
+	  == CTF_ERR)
+	goto err_target;
+      break;
+
+    case CTF_K_ENUM:
+      {
+	int val;
+	erm = "enum";
+	if ((new_type = ctf_add_enum (target, isroot, name)) == CTF_ERR)
+	  goto err_input;				/* errno is set for us.  */
+
+	while ((name = ctf_enum_next (input, type, &i, &val)) != NULL)
+	  {
+	    if (ctf_add_enumerator (target, new_type, name, val) < 0)
+	      {
+		ctf_err_warn (target, 0, "%s (%i): cannot add enumeration "
+			      "value %s from input type %lx: %s",
+			      ctf_link_input_name (input), input_num, name,
+			      type, ctf_errmsg (ctf_errno (target)));
+		ctf_next_destroy (i);
+		return ctf_set_errno (output, ctf_errno (target));
+	      }
+	  }
+	if (ctf_errno (input) != ECTF_NEXT_END)
+	  goto err_input;
+	break;
+      }
+
+    case CTF_K_TYPEDEF:
+      erm = "typedef";
+
+      ref = ctf_type_reference (input, type);
+      if ((ref = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+					 parents, input, input_num,
+					 ref)) == CTF_ERR)
+	goto err_input;				/* errno is set for us.  */
+
+      if ((new_type = ctf_add_typedef (target, isroot, name, ref)) == CTF_ERR)
+	goto err_target;			/* errno is set for us.  */
+      break;
+
+    case CTF_K_VOLATILE:
+    case CTF_K_CONST:
+    case CTF_K_RESTRICT:
+    case CTF_K_POINTER:
+      erm = "pointer or cvr-qual";
+
+      ref = ctf_type_reference (input, type);
+      if ((ref = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+					 parents, input, input_num,
+					 ref)) == CTF_ERR)
+	goto err_input;				/* errno is set for us.  */
+
+      if ((new_type = ctf_add_reftype (target, isroot, ref, kind)) == CTF_ERR)
+	goto err_target;			/* errno is set for us.  */
+      break;
+
+    case CTF_K_SLICE:
+      erm = "slice";
+
+      if (ctf_type_encoding (input, type, &ep) < 0)
+	goto err_input;				/* errno is set for us.  */
+
+      ref = ctf_type_reference (input, type);
+      if ((ref = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+					 parents, input, input_num,
+					 ref)) == CTF_ERR)
+	goto err_input;
+
+      if ((new_type = ctf_add_slice (target, isroot, ref, &ep)) == CTF_ERR)
+	goto err_target;
+      break;
+
+    case CTF_K_ARRAY:
+      {
+	ctf_arinfo_t ar;
+
+	erm = "array info";
+	if (ctf_array_info (input, type, &ar) < 0)
+	  goto err_input;
+
+	ar.ctr_contents = ctf_dedup_id_to_target (output, target, inputs,
+						  ninputs, parents, input,
+						  input_num, ar.ctr_contents);
+	ar.ctr_index = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+					       parents, input, input_num,
+					       ar.ctr_index);
+
+	if (ar.ctr_contents == CTF_ERR || ar.ctr_index == CTF_ERR)
+	  goto err_input;
+
+	if ((new_type = ctf_add_array (target, isroot, &ar)) == CTF_ERR)
+	  goto err_target;
+
+	break;
+      }
+
+    case CTF_K_FUNCTION:
+      {
+	ctf_funcinfo_t fi;
+	ctf_id_t *args;
+	uint32_t j;
+
+	erm = "function";
+	if (ctf_func_type_info (input, type, &fi) < 0)
+	  goto err_input;
+
+	fi.ctc_return = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+						parents, input, input_num,
+						fi.ctc_return);
+	if (fi.ctc_return == CTF_ERR)
+	  goto err_input;
+
+	if ((args = calloc (fi.ctc_argc, sizeof (ctf_id_t))) == NULL)
+	  {
+	    ctf_set_errno (input, ENOMEM);
+	    goto err_input;
+	  }
+
+	erm = "function args";
+	if (ctf_func_type_args (input, type, fi.ctc_argc, args) < 0)
+	  {
+	    free (args);
+	    goto err_input;
+	  }
+
+	for (j = 0; j < fi.ctc_argc; j++)
+	  {
+	    args[j] = ctf_dedup_id_to_target (output, target, inputs, ninputs,
+					      parents, input, input_num,
+					      args[j]);
+	    if (args[j] == CTF_ERR)
+	      goto err_input;
+	  }
+
+	if ((new_type = ctf_add_function (target, isroot,
+					  &fi, args)) == CTF_ERR)
+	  {
+	    free (args);
+	    goto err_target;
+	  }
+	free (args);
+	break;
+      }
+
+    case CTF_K_STRUCT:
+    case CTF_K_UNION:
+      {
+	size_t size = ctf_type_size (input, type);
+	void *out_id;
+	/* Insert the structure itself, so other types can refer to it.  */
+
+	erm = "structure/union";
+	if (kind == CTF_K_STRUCT)
+	  new_type = ctf_add_struct_sized (target, isroot, name, size);
+	else
+	  new_type = ctf_add_union_sized (target, isroot, name, size);
+
+	if (new_type == CTF_ERR)
+	  goto err_target;
+
+	out_id = CTF_DEDUP_GID (output, output_num, new_type);
+	ctf_dprintf ("%i: Noting need to emit members of %p -> %p\n", depth,
+		     id, out_id);
+	/* Record the need to emit the members of this structure later.  */
+	if (ctf_dynhash_insert (d->cd_emission_struct_members, id, out_id) < 0)
+	  goto err_target;
+	break;
+      }
+    default:
+      ctf_err_warn (output, 0, "%s: unknown type kind for input type %lx",
+		    ctf_link_input_name (input), type);
+      return -1;
+    }
+
+  if (!emission_hashed
+      && new_type != 0
+      && ctf_dynhash_cinsert (target->ctf_dedup.cd_output_emission_hashes,
+			      hval, (void *) new_type) < 0)
+    {
+      ctf_err_warn (output, 0, "out of memory tracking deduplicated "
+		    "global type IDs");
+	return ctf_set_errno (output, ENOMEM);
+    }
+
+  if (!emission_hashed && new_type != 0)
+    ctf_dprintf ("%i: Inserted %s, %i/%lx -> %lx into emission hash for "
+		 "target %p (%s)\n", depth, hval, input_num, type, new_type,
+		 (void *) target, ctf_link_input_name (target));
+
+  return 0;
+
+ oom_hash:
+  ctf_err_warn (output, 0, "out of memory creating emission-tracking hashes");
+  return ctf_set_errno (output, ENOMEM);
+
+ err_input:
+  ctf_err_warn (output, 0, "%s (%i): while emitting deduplicated %s, error "
+		"getting input type %lx: %s", ctf_link_input_name (input),
+		input_num,erm, type, ctf_errmsg (ctf_errno (input)));
+  ctf_set_errno (output, ctf_errno (input));
+  return -1;
+ err_target:
+  ctf_err_warn (output, 0, "%s (%i): while emitting deduplicated %s, error "
+		"emitting target type from input type %lx: %s",
+		ctf_link_input_name (input), input_num, erm, type,
+		ctf_errmsg (ctf_errno (target)));
+  ctf_set_errno (output, ctf_errno (target));
+  return -1;
+}
+
+/* Traverse the cd_emission_struct_members and emit the members of all
+   structures and unions.  All other types are emitted and complete by this
+   point.  */
+
+static int
+ctf_dedup_emit_struct_members (ctf_file_t *output, ctf_file_t **inputs,
+			       uint32_t ninputs, uint32_t *parents)
+{
+  ctf_dedup_t *d = &output->ctf_dedup;
+  ctf_next_t *i = NULL;
+  void *input_id, *target_id;
+  int err;
+  ctf_file_t *err_fp, *input_fp;
+  int input_num;
+  ctf_id_t err_type;
+
+  while ((err = ctf_dynhash_next (d->cd_emission_struct_members, &i,
+				  &input_id, &target_id)) == 0)
+    {
+      ctf_next_t *j = NULL;
+      ctf_file_t *target;
+      uint32_t target_num;
+      ctf_id_t input_type, target_type;
+      ssize_t offset;
+      ctf_id_t membtype;
+      const char *name;
+
+      input_num = CTF_DEDUP_GID_TO_INPUT (input_id);
+      input_fp = inputs[input_num];
+      input_type = CTF_DEDUP_GID_TO_TYPE (input_id);
+
+      /* The output is either -1 (for the shared, parent output dict) or the
+	 number of the corresponding input.  */
+      target_num = CTF_DEDUP_GID_TO_INPUT (target_id);
+      if (target_num == (uint32_t) -1)
+	target = output;
+      else
+	{
+	  target = inputs[target_num]->ctf_dedup.cd_output;
+	  if (!ctf_assert (output, target))
+	    {
+	      err_fp = output;
+	      err_type = input_type;
+	      goto err_target;
+	    }
+	}
+      target_type = CTF_DEDUP_GID_TO_TYPE (target_id);
+
+      while ((offset = ctf_member_next (input_fp, input_type, &j, &name,
+					&membtype)) >= 0)
+	{
+	  err_fp = target;
+	  err_type = target_type;
+	  if ((membtype = ctf_dedup_id_to_target (output, target, inputs,
+						  ninputs, parents, input_fp,
+						  input_num,
+						  membtype)) == CTF_ERR)
+	    {
+	      ctf_next_destroy (j);
+	      goto err_target;
+	    }
+
+	  if (name == NULL)
+	    name = "";
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+	  ctf_dprintf ("Emitting %s, offset %zi\n", name, offset);
+#endif
+	  if (ctf_add_member_offset (target, target_type, name,
+				     membtype, offset) < 0)
+	    {
+	      ctf_next_destroy (j);
+	      goto err_target;
+	    }
+	}
+      if (ctf_errno (input_fp) != ECTF_NEXT_END)
+	{
+	  err = ctf_errno (input_fp);
+	  ctf_next_destroy (i);
+	  goto iterr;
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    goto iterr;
+
+  return 0;
+ err_target:
+  ctf_next_destroy (i);
+  ctf_err_warn (output, 0, "%s (%i): error emitting members for structure "
+		"type %lx: %s", ctf_link_input_name (input_fp), input_num,
+		err_type, ctf_errmsg (ctf_errno (err_fp)));
+  ctf_set_errno (output, ctf_errno (err_fp));
+  return -1;
+ iterr:
+  ctf_err_warn (output, 0, "iteration failure emitting structure members: %s",
+		ctf_errmsg (err));
+  ctf_set_errno (output, err);
+  return -1;
+}
+
+/* Populate the type mapping used by the types in one FP (which must be an input
+   dict containing a non-null cd_output resulting from a ctf_dedup_emit_type
+   walk).  */
+static int
+ctf_dedup_populate_type_mapping (ctf_file_t *shared, ctf_file_t *fp,
+				 ctf_file_t **inputs)
+{
+  ctf_dedup_t *d = &shared->ctf_dedup;
+  ctf_file_t *output = fp->ctf_dedup.cd_output;
+  const void *k, *v;
+  ctf_next_t *i = NULL;
+  int err;
+
+  /* The shared dict (the output) stores its types in the fp itself, not in a
+     separate cd_output dict.  */
+  if (shared == fp)
+    output = fp;
+
+  /* There may be no types to emit at all, or all the types in this TU may be
+     shared.  */
+  if (!output || !output->ctf_dedup.cd_output_emission_hashes)
+    return 0;
+
+  while ((err = ctf_dynhash_cnext (output->ctf_dedup.cd_output_emission_hashes,
+				  &i, &k, &v)) == 0)
+    {
+      const char *hval = (const char *) k;
+      ctf_id_t id_out = (ctf_id_t) v;
+      ctf_next_t *j = NULL;
+      ctf_dynset_t *type_ids;
+      const void *id;
+
+      type_ids = ctf_dynhash_lookup (d->cd_output_mapping, hval);
+      if (!ctf_assert (shared, type_ids))
+	return -1;
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+      ctf_dprintf ("Traversing emission hash: hval %s\n", hval);
+#endif
+
+      while ((err = ctf_dynset_cnext (type_ids, &j, &id)) == 0)
+	{
+	  ctf_file_t *input = inputs[CTF_DEDUP_GID_TO_INPUT (id)];
+	  ctf_id_t id_in = CTF_DEDUP_GID_TO_TYPE (id);
+
+#ifdef ENABLE_LIBCTF_HASH_DEBUGGING
+	  ctf_dprintf ("Adding mapping from %i/%lx to %lx\n",
+		       CTF_DEDUP_GID_TO_INPUT (id), id_in, id_out);
+#endif
+	  ctf_add_type_mapping (input, id_in, output, id_out);
+	}
+      if (err != ECTF_NEXT_END)
+	{
+	  ctf_next_destroy (i);
+	  goto err;
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    goto err;
+
+  return 0;
+
+ err:
+  ctf_err_warn (shared, 0, "iteration error populating the type mapping: %s",
+		ctf_errmsg (err));
+  return ctf_set_errno (shared, err);
+}
+
+/* Populate the type mapping machinery used by the rest of the linker,
+   by ctf_add_type, etc.  */
+static int
+ctf_dedup_populate_type_mappings (ctf_file_t *output, ctf_file_t **inputs,
+				  uint32_t ninputs)
+{
+  size_t i;
+
+  if (ctf_dedup_populate_type_mapping (output, output, inputs) < 0)
+    {
+      ctf_err_warn (output, 0, "cannot populate type mappings for shared "
+		    "CTF dict: %s", ctf_errmsg (ctf_errno (output)));
+      return -1;				/* errno is set for us.  */
+    }
+
+  for (i = 0; i < ninputs; i++)
+    {
+      if (ctf_dedup_populate_type_mapping (output, inputs[i], inputs) < 0)
+	{
+	  ctf_err_warn (output, 0, "cannot populate type mappings for per-CU "
+			"CTF dict: %s", ctf_errmsg (ctf_errno (inputs[i])));
+	  return ctf_set_errno (output, ctf_errno (inputs[i]));
+	}
+    }
+
+  return 0;
+}
+
+/* Emit deduplicated types into the outputs.  The shared type repository is
+   OUTPUT, on which the ctf_dedup function must have already been called.  The
+   PARENTS array contains the INPUTS index of the parent dict for every child
+   dict at the corresponding index in the INPUTS (for non-child dicts, the value
+   is undefined).
+
+   Return an array of fps with content emitted into them (starting with OUTPUT,
+   which is the parent of all others, then all the newly-generated outputs).
+
+   If CU_MAPPED is set, this is a first pass for a link with a non-empty CU
+   mapping: only one output will result.  */
+
+ctf_file_t **
+ctf_dedup_emit (ctf_file_t *output, ctf_file_t **inputs, uint32_t ninputs,
+		uint32_t *parents, uint32_t *noutputs, int cu_mapped)
+{
+  size_t num_outputs = 1;		/* Always at least one output: us.  */
+  ctf_file_t **outputs;
+  ctf_file_t **walk;
+  size_t i;
+
+  ctf_dprintf ("Triggering emission.\n");
+  if (ctf_dedup_walk_output_mapping (output, inputs, ninputs, parents,
+				     ctf_dedup_emit_type, &cu_mapped) < 0)
+    return NULL;				/* errno is set for us.  */
+
+  ctf_dprintf ("Populating struct members.\n");
+  if (ctf_dedup_emit_struct_members (output, inputs, ninputs, parents) < 0)
+    return NULL;				/* errno is set for us.  */
+
+  if (ctf_dedup_populate_type_mappings (output, inputs, ninputs) < 0)
+    return NULL;				/* errno is set for us.  */
+
+  for (i = 0; i < ninputs; i++)
+    {
+      if (inputs[i]->ctf_dedup.cd_output)
+	num_outputs++;
+    }
+
+  if (!ctf_assert (output, !cu_mapped || (cu_mapped && num_outputs == 1)))
+    return NULL;
+
+  if ((outputs = calloc (num_outputs, sizeof (ctf_file_t *))) == NULL)
+    {
+      ctf_err_warn (output, 0, "out of memory allocating link outputs array");
+      ctf_set_errno (output, ENOMEM);
+      return NULL;
+    }
+  *noutputs = num_outputs;
+
+  walk = outputs;
+  *walk = output;
+  output->ctf_refcnt++;
+  walk++;
+
+  for (i = 0; i < ninputs; i++)
+    {
+      if (inputs[i]->ctf_dedup.cd_output)
+	{
+	  *walk = inputs[i]->ctf_dedup.cd_output;
+	  inputs[i]->ctf_dedup.cd_output = NULL;
+	  walk++;
+	}
+    }
+
+  ctf_dedup_fini (output, outputs, num_outputs);
+  return outputs;
+}
diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index b9816473a8b..76572ff96e4 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -116,6 +116,28 @@ ctf_hash_eq_type_key (const void *a, const void *b)
     && (key_a->cltk_idx == key_b->cltk_idx);
 }
 
+/* Hash a type_id_key.  */
+unsigned int
+ctf_hash_type_id_key (const void *ptr)
+{
+  ctf_helem_t *hep = (ctf_helem_t *) ptr;
+  ctf_type_id_key_t *k = (ctf_type_id_key_t *) hep->key;
+
+  return htab_hash_pointer ((void *) (uintptr_t) k->ctii_input_num)
+    + 59 * htab_hash_pointer ((void *) k->ctii_type);
+}
+
+int
+ctf_hash_eq_type_id_key (const void *a, const void *b)
+{
+  ctf_helem_t *hep_a = (ctf_helem_t *) a;
+  ctf_helem_t *hep_b = (ctf_helem_t *) b;
+  ctf_type_id_key_t *key_a = (ctf_type_id_key_t *) hep_a->key;
+  ctf_type_id_key_t *key_b = (ctf_type_id_key_t *) hep_b->key;
+
+  return (key_a->ctii_input_num == key_b->ctii_input_num)
+    && (key_a->ctii_type == key_b->ctii_type);
+}
 
 /* Hash and eq functions for the dynset.  Most of these can just use the
    underlying hashtab functions directly.   */
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 4dea3f018f0..41e9f448c59 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -247,6 +247,106 @@ typedef struct ctf_link_type_key
   ctf_id_t cltk_idx;
 } ctf_link_type_key_t;
 
+/* The structure used as the key in a cd_id_to_file_t on 32-bit platforms.  */
+typedef struct ctf_type_id_key
+{
+  int ctii_input_num;
+  ctf_id_t ctii_type;
+} ctf_type_id_key_t;
+
+/* Deduplicator state.
+
+   The dedup state below uses three terms consistently. A "hash" is a
+   ctf_dynhash_t; a "hash value" is the hash value of a type as returned by
+   ctf_dedup_hash_type; a "global type ID" or "global ID" is a packed-together
+   reference to a single ctf_file_t (by array index in an array of inputs) and
+   ctf_id_t, i.e. a single instance of some hash value in some input.
+
+   The deduplication algorithm takes a bunch of inputs and yields a single
+   shared "output" and possibly many outputs corresponding to individual inputs
+   that still contain types after sharing of unconflicted types.  Almost all
+   deduplicator state is stored in the struct ctf_dedup in the output, though a
+   (very) few things are stored in inputs for simplicity's sake, usually if they
+   are linking together things within the scope of a single TU.
+
+   Flushed at the end of every ctf_dedup run.  */
+
+typedef struct ctf_dedup
+{
+  /* The CTF linker flags in force for this dedup run.  */
+  int cd_link_flags;
+
+  /* On 32-bit platforms only, a hash of global type IDs, in the form of
+     a ctf_link_type_id_key_t.  */
+  ctf_dynhash_t *cd_id_to_file_t;
+
+  /* Atoms tables of decorated names: maps undecorated name to decorated name.
+     (The actual allocations are in the CTF file for the former and the real
+     atoms table for the latter).  Uses the same namespaces as ctf_lookups,
+     below, but has no need for null-termination.  */
+  ctf_dynhash_t *cd_decorated_names[4];
+
+  /* Map type names to a hash from type hash value -> number of times each value
+     has appeared.  */
+  ctf_dynhash_t *cd_name_counts;
+
+  /* Map global type IDs to type hash values.  Used to determine if types are
+     already hashed without having to recompute their hash values again, and to
+     link types together at later stages.  Forwards that are peeked through to
+     structs and unions are not represented in here, so lookups that might be
+     such a type (in practice, all lookups) must go via cd_replaced_types first
+     to take this into account.  Discarded before each rehashing.  */
+  ctf_dynhash_t *cd_type_hashes;
+
+  /* Maps from the names of structs/unions/enums to a a single GID which is the
+     only appearance of that type in any input: if it appears in more than one
+     input, a value which is a GID with an input_num of -1 appears.  Used in
+     share-duplicated link mode link modes to determine whether structs/unions
+     can be cited from multiple TUs.  Only populated in that link mode.  */
+  ctf_dynhash_t *cd_struct_origin;
+
+  /* Maps type hash values to a set of hash values of the types that cite them:
+     i.e., pointing backwards up the type graph.  Used for recursive conflict
+     marking.  Citations from tagged structures, unions, and forwards do not
+     appear in this graph.  */
+  ctf_dynhash_t *cd_citers;
+
+  /* Maps type hash values to input global type IDs.  The value is a set (a
+     hash) of global type IDs.  Discarded before each rehashing.  The result of
+     the ctf_dedup function.  */
+  ctf_dynhash_t *cd_output_mapping;
+
+  /* A map giving the GID of the first appearance of each type for each type
+     hash value.  */
+  ctf_dynhash_t *cd_output_first_gid;
+
+  /* Used to ensure that we never try to map a single type ID to more than one
+     hash.  */
+  ctf_dynhash_t *cd_output_mapping_guard;
+
+  /* Maps the global type IDs of structures in input TUs whose members still
+     need emission to the global type ID of the already-emitted target type
+     (which has no members yet) in the appropriate target.  Uniquely, the latter
+     ID represents a *target* ID (i.e. the cd_output_mapping of some specified
+     input): we encode the shared (parent) dict with an ID of -1.  */
+  ctf_dynhash_t *cd_emission_struct_members;
+
+  /* A set (a hash) of hash values of conflicting types.  */
+  ctf_dynset_t *cd_conflicting_types;
+
+  /* Maps type hashes to ctf_id_t's in this dictionary.  Populated only at
+     emission time, in the dictionary where emission is taking place.  */
+  ctf_dynhash_t *cd_output_emission_hashes;
+
+  /* Maps the decorated names of conflicted cross-TU forwards that were forcibly
+     emitted in this TU to their emitted ctf_id_ts.  Populated only at emission
+     time, in the dictionary where emission is taking place. */
+  ctf_dynhash_t *cd_output_emission_conflicted_forwards;
+
+  /* Points to the output counterpart of this input dictionary, at emission
+     time.  */
+  ctf_file_t *cd_output;
+} ctf_dedup_t;
 
 /* The ctf_file is the structure used to represent a CTF container to library
    clients, who see it only as an opaque pointer.  Modifications can therefore
@@ -346,6 +446,18 @@ struct ctf_file
   void *ctf_link_variable_filter_arg;           /* Argument for it. */
 
   ctf_dynhash_t *ctf_add_processing; /* Types ctf_add_type is working on now.  */
+
+  /* Atoms table for dedup string storage.  All strings in the ctf_dedup_t are
+     stored here.  Only the _alloc copy is allocated or freed: the
+     ctf_dedup_atoms may be pointed to some other CTF dict, to share its atoms.
+     We keep the atoms table outside the ctf_dedup so that atoms can be
+     preserved across multiple similar links, such as when doing cu-mapped
+     links.  */
+  ctf_dynset_t *ctf_dedup_atoms;
+  ctf_dynset_t *ctf_dedup_atoms_alloc;
+
+  ctf_dedup_t ctf_dedup;	  /* Deduplicator state.  */
+
   char *ctf_tmp_typeslice;	  /* Storage for slicing up type names.  */
   size_t ctf_tmp_typeslicelen;	  /* Size of the typeslice.  */
   void *ctf_specific;		  /* Data for ctf_get/setspecific().  */
@@ -451,11 +563,13 @@ typedef unsigned int (*ctf_hash_fun) (const void *ptr);
 extern unsigned int ctf_hash_integer (const void *ptr);
 extern unsigned int ctf_hash_string (const void *ptr);
 extern unsigned int ctf_hash_type_key (const void *ptr);
+extern unsigned int ctf_hash_type_id_key (const void *ptr);
 
 typedef int (*ctf_hash_eq_fun) (const void *, const void *);
 extern int ctf_hash_eq_integer (const void *, const void *);
 extern int ctf_hash_eq_string (const void *, const void *);
 extern int ctf_hash_eq_type_key (const void *, const void *);
+extern int ctf_hash_eq_type_id_key (const void *, const void *);
 
 extern int ctf_dynset_eq_string (const void *, const void *);
 
@@ -526,11 +640,24 @@ extern int ctf_dvd_insert (ctf_file_t *, ctf_dvdef_t *);
 extern void ctf_dvd_delete (ctf_file_t *, ctf_dvdef_t *);
 extern ctf_dvdef_t *ctf_dvd_lookup (const ctf_file_t *, const char *);
 
+extern ctf_id_t ctf_add_encoded (ctf_file_t *, uint32_t, const char *,
+				 const ctf_encoding_t *, uint32_t kind);
+extern ctf_id_t ctf_add_reftype (ctf_file_t *, uint32_t, ctf_id_t,
+				 uint32_t kind);
+
 extern void ctf_add_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type,
 				  ctf_file_t *dst_fp, ctf_id_t dst_type);
 extern ctf_id_t ctf_type_mapping (ctf_file_t *src_fp, ctf_id_t src_type,
 				  ctf_file_t **dst_fp);
 
+extern int ctf_dedup_atoms_init (ctf_file_t *);
+extern int ctf_dedup (ctf_file_t *, ctf_file_t **, uint32_t ninputs,
+		      uint32_t *parents, int cu_mapped);
+extern void ctf_dedup_fini (ctf_file_t *, ctf_file_t **, uint32_t);
+extern ctf_file_t **ctf_dedup_emit (ctf_file_t *, ctf_file_t **,
+				    uint32_t ninputs, uint32_t *parents,
+				    uint32_t *noutputs, int cu_mapped);
+
 extern void ctf_decl_init (ctf_decl_t *);
 extern void ctf_decl_fini (ctf_decl_t *);
 extern void ctf_decl_push (ctf_decl_t *, ctf_file_t *, ctf_id_t);
diff --git a/libctf/ctf-open.c b/libctf/ctf-open.c
index 285e0e0ff71..fee7789a308 100644
--- a/libctf/ctf-open.c
+++ b/libctf/ctf-open.c
@@ -1719,6 +1719,8 @@ ctf_file_close (ctf_file_t *fp)
   ctf_dynhash_destroy (fp->ctf_link_in_cu_mapping);
   ctf_dynhash_destroy (fp->ctf_link_out_cu_mapping);
   ctf_dynhash_destroy (fp->ctf_add_processing);
+  ctf_dedup_fini (fp, NULL, 0);
+  ctf_dynset_destroy (fp->ctf_dedup_atoms_alloc);
 
   for (err = ctf_list_next (&fp->ctf_errs_warnings); err != NULL; err = nerr)
     {
-- 
2.27.0.247.g3dff7de930

[PATCH 51/59] libctf, link: add CTF_LINK_OMIT_VARIABLES_SECTION

[Nick Alcock]

This flag (not used anywhere yet) causes the variables section to be
omitted from the output CTF dict.

include/
	* ctf-api.h (CTF_LINK_OMIT_VARIABLES_SECTION): New.
libctf/
	* ctf-link.c (ctf_link_one_input_archive_member): Check
	CTF_LINK_OMIT_VARIABLES_SECTION.
---
 include/ctf-api.h | 3 +++
 libctf/ctf-link.c | 3 ++-
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index c3683db0996..f034f9859dd 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -90,6 +90,9 @@ typedef struct ctf_link_sym
    emitted into them.  */
 #define CTF_LINK_EMPTY_CU_MAPPINGS 0x4
 
+/* Omit the content of the variables section.  */
+#define CTF_LINK_OMIT_VARIABLES_SECTION 0x8
+
 /* Symbolic names for CTF sections.  */
 
 typedef enum ctf_sect_names
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 886106cb478..043d6a2f530 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -685,7 +685,8 @@ ctf_link_one_input_archive_member (ctf_file_t *in_fp, const char *name, void *ar
   arg->in_fp = in_fp;
 
   if ((err = ctf_type_iter_all (in_fp, ctf_link_one_type, arg)) > -1)
-    err = ctf_variable_iter (in_fp, ctf_link_one_variable, arg);
+    if (!(in_fp->ctf_link_flags & CTF_LINK_OMIT_VARIABLES_SECTION))
+      err = ctf_variable_iter (in_fp, ctf_link_one_variable, arg);
 
   arg->in_input_cu_file = 0;
 
-- 
2.27.0.247.g3dff7de930

[PATCH 52/59] libctf, link: tie in the deduplicating linker

[Nick Alcock]

This fairly intricate commit connects up the CTF linker machinery (which
operates in terms of ctf_archive_t's on ctf_link_inputs ->
ctf_link_outputs) to the deduplicator (which operates in terms of arrays
of ctf_file_t's, all the archives exploded).

The nondeduplicating linker is retained, but is not called unless the
CTF_LINK_NONDEDUP flag is passed in (which ld never does), or the
environment variable LD_NO_CTF_DEDUP is set.  Eventually, once we have
confidence in the much-more-complex deduplicating linker, I hope the
nondeduplicating linker can be removed.

In brief, what this does is traverses each input archive in
ctf_link_inputs, opening every member (if not already open) and tying
child dicts to their parents, shoving them into an array and
constructing a corresponding parents array that tells the deduplicator
which dict is the parent of which child.  We then call ctf_dedup and
ctf_dedup_emit with that array of inputs, taking the outputs that result
and putting them into ctf_link_outputs where the rest of the CTF linker
expects to find them, then linking in the variables just as is done by
the nondeduplicating linker.

It also implements much of the CU-mapping side of things.  The problem
CU-mapping introduces is that if you map many input CUs into one output,
this is saying that you want many translation units to produce at most
one child dict if conflicting types are found in any of them.  This
means you can suddenly have multiple distinct types with the same name
in the same dict, which libctf cannot really represent because it's not
something you can do with C translation units.

The deduplicator machinery already committed does as best it can with
these, hiding types with conflicting names rather than making child
dicts out of them: but we still need to call it.  This is done similarly
to the main link, taking the inputs (one CU output at a time),
deduplicating them, taking the output and making it an input to the
final link.  Two (significant) optimizations are done: we share atoms
tables between all these links and the final link (so e.g. all type hash
values are shared, all decorated type names, etc); and any CU-mapped
links with only one input (and no child dicts) doesn't need to do
anything other than renaming the CU: the CU-mapped link phase can be
skipped for it.  Put together, large CU-mapped links can save 50% of
their memory usage and about as much time (and the memory usage for
CU-mapped links is significant, because all those output CUs have to
have all their types stored in memory all at once).

include/
	* ctf-api.h (CTF_LINK_NONDEDUP): New, turn off the
	deduplicator.
libctf/
	* ctf-impl.h (ctf_list_splice): New.
	* ctf-util.h (ctf_list_splice): Likewise.
	* ctf-link.c (link_sort_inputs_cb_arg_t): Likewise.
	(ctf_link_sort_inputs): Likewise.
	(ctf_link_deduplicating_count_inputs): Likewise.
	(ctf_link_deduplicating_open_inputs): Likewise.
	(ctf_link_deduplicating_close_inputs): Likewise.
	(ctf_link_deduplicating_variables): Likewise.
	(ctf_link_deduplicating_per_cu): Likewise.
	(ctf_link_deduplicating): Likewise.
	(ctf_link): Call it.
---
 include/ctf-api.h |   3 +
 libctf/ctf-impl.h |   1 +
 libctf/ctf-link.c | 659 +++++++++++++++++++++++++++++++++++++++++++++-
 libctf/ctf-util.c |  20 ++
 4 files changed, 681 insertions(+), 2 deletions(-)

diff --git a/include/ctf-api.h b/include/ctf-api.h
index f034f9859dd..77ea5cd0ef8 100644
--- a/include/ctf-api.h
+++ b/include/ctf-api.h
@@ -86,6 +86,9 @@ typedef struct ctf_link_sym
 /* Share only types that are used by multiple inputs.  */
 #define CTF_LINK_SHARE_DUPLICATED 0x1
 
+/* Do a nondeduplicating link.  */
+#define CTF_LINK_NONDEDUP 0x2
+
 /* Create empty outputs for all registered CU mappings even if no types are
    emitted into them.  */
 #define CTF_LINK_EMPTY_CU_MAPPINGS 0x4
diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 41e9f448c59..06b98da2a9d 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -629,6 +629,7 @@ extern char *ctf_sha1_fini (ctf_sha1_t *, char *);
 extern void ctf_list_append (ctf_list_t *, void *);
 extern void ctf_list_prepend (ctf_list_t *, void *);
 extern void ctf_list_delete (ctf_list_t *, void *);
+extern void ctf_list_splice (ctf_list_t *, ctf_list_t *);
 extern int ctf_list_empty_p (ctf_list_t *lp);
 
 extern int ctf_dtd_insert (ctf_file_t *, ctf_dtdef_t *, int flag, int kind);
diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index 043d6a2f530..fd7298749cd 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -819,6 +819,658 @@ ctf_link_one_input_archive (void *key, void *value, void *arg_)
   ctf_link_close_one_input_archive (key, value, NULL);
 }
 
+typedef struct link_sort_inputs_cb_arg
+{
+  int is_cu_mapped;
+  ctf_file_t *fp;
+} link_sort_inputs_cb_arg_t;
+
+/* Sort the inputs by N (the link order).  For CU-mapped links, this is a
+   mapping of input to output name, not a mapping of input name to input
+   ctf_link_input_t: compensate accordingly.  */
+static int
+ctf_link_sort_inputs (const ctf_next_hkv_t *one, const ctf_next_hkv_t *two,
+		      void *arg)
+{
+  ctf_link_input_t *input_1;
+  ctf_link_input_t *input_2;
+  link_sort_inputs_cb_arg_t *cu_mapped = (link_sort_inputs_cb_arg_t *) arg;
+
+  if (!cu_mapped || !cu_mapped->is_cu_mapped)
+    {
+      input_1 = (ctf_link_input_t *) one->hkv_value;
+      input_2 = (ctf_link_input_t *) two->hkv_value;
+    }
+  else
+    {
+      const char *name_1 = (const char *) one->hkv_key;
+      const char *name_2 = (const char *) two->hkv_key;
+
+      input_1 = ctf_dynhash_lookup (cu_mapped->fp->ctf_link_inputs, name_1);
+      input_2 = ctf_dynhash_lookup (cu_mapped->fp->ctf_link_inputs, name_2);
+
+      /* There is no guarantee that CU-mappings actually have corresponding
+	 inputs: the relative ordering in that case is unimportant.  */
+      if (!input_1)
+	return -1;
+      if (!input_2)
+	return 1;
+    }
+
+  if (input_1->n < input_2->n)
+    return -1;
+  else if (input_1->n > input_2->n)
+    return 1;
+  else
+    return 0;
+}
+
+/* Count the number of input dicts in the ctf_link_inputs, or that subset of the
+   ctf_link_inputs given by CU_NAMES if set.  Return the number of input dicts,
+   and optionally the name and ctf_link_input_t of the single input archive if
+   only one exists (no matter how many dicts it contains).  */
+static ssize_t
+ctf_link_deduplicating_count_inputs (ctf_file_t *fp, ctf_dynhash_t *cu_names,
+				     ctf_link_input_t **only_one_input)
+{
+  ctf_dynhash_t *inputs = fp->ctf_link_inputs;
+  ctf_next_t *i = NULL;
+  void *name, *input;
+  ctf_link_input_t *one_input = NULL;
+  const char *one_name = NULL;
+  ssize_t count = 0, narcs = 0;
+  int err;
+
+  if (cu_names)
+    inputs = cu_names;
+
+  while ((err = ctf_dynhash_next (inputs, &i, &name, &input)) == 0)
+    {
+      ssize_t one_count;
+
+      one_name = (const char *) name;
+      /* If we are processing CU names, get the real input.  */
+      if (cu_names)
+	one_input = ctf_dynhash_lookup (fp->ctf_link_inputs, one_name);
+      else
+	one_input = (ctf_link_input_t *) input;
+
+      if (!one_input)
+	continue;
+
+      one_count = ctf_link_lazy_open (fp, one_input);
+
+      if (one_count < 0)
+	{
+	  ctf_next_destroy (i);
+	  return -1;				/* errno is set for us.  */
+	}
+
+      count += one_count;
+      narcs++;
+    }
+  if (err != ECTF_NEXT_END)
+    {
+      ctf_err_warn (fp, 0, "Iteration error counting deduplicating CTF link "
+		    "inputs: %s", ctf_errmsg (err));
+      ctf_set_errno (fp, err);
+      return -1;
+    }
+
+  if (!count)
+    return 0;
+
+  if (narcs == 1)
+    {
+      if (only_one_input)
+	*only_one_input = one_input;
+    }
+  else if (only_one_input)
+    *only_one_input = NULL;
+
+  return count;
+}
+
+/* Allocate and populate an inputs array big enough for a given set of inputs:
+   either a specific set of CU names (those from that set found in the
+   ctf_link_inputs), or the entire ctf_link_inputs (if cu_names is not set).
+   The number of inputs (from ctf_link_deduplicating_count_inputs, above) is
+   passed in NINPUTS: an array of uint32_t containing parent pointers
+   (corresponding to those members of the inputs that have parents) is allocated
+   and returned in PARENTS.
+
+   The inputs are *archives*, not files: the archive can have multiple members
+   if it is the result of a previous incremental link.  We want to add every one
+   in turn, including the shared parent.  (The dedup machinery knows that a type
+   used by a single dictionary and its parent should not be shared in
+   CTF_LINK_SHARE_DUPLICATED mode.)
+
+   If no inputs exist that correspond to these CUs, return NULL with the errno
+   set to ECTF_NOCTFDATA.  */
+static ctf_file_t **
+ctf_link_deduplicating_open_inputs (ctf_file_t *fp, ctf_dynhash_t *cu_names,
+				    ssize_t ninputs, uint32_t **parents)
+{
+  ctf_dynhash_t *inputs = fp->ctf_link_inputs;
+  ctf_next_t *i = NULL;
+  void *name, *input;
+  link_sort_inputs_cb_arg_t sort_arg;
+  ctf_file_t **dedup_inputs = NULL;
+  ctf_file_t **walk;
+  uint32_t *parents_ = NULL;
+  int err;
+
+  if (cu_names)
+    inputs = cu_names;
+
+  if ((dedup_inputs = calloc (ninputs, sizeof (ctf_file_t *))) == NULL)
+    goto oom;
+
+  if ((parents_ = calloc (ninputs, sizeof (uint32_t))) == NULL)
+    goto oom;
+
+  walk = dedup_inputs;
+
+  /* Counting done: push every input into the array, in the order they were
+     passed to ctf_link_add_ctf (and ultimately ld).  */
+
+  sort_arg.is_cu_mapped = (cu_names != NULL);
+  sort_arg.fp = fp;
+
+  while ((err = ctf_dynhash_next_sorted (inputs, &i, &name, &input,
+					 ctf_link_sort_inputs, &sort_arg)) == 0)
+    {
+      const char *one_name = (const char *) name;
+      ctf_link_input_t *one_input;
+      ctf_file_t *one_fp;
+      ctf_file_t *parent_fp = NULL;
+      uint32_t parent_i;
+      ctf_next_t *j = NULL;
+
+      /* If we are processing CU names, get the real input.  All the inputs
+	 will have been opened, if they contained any CTF at all.  */
+      if (cu_names)
+	one_input = ctf_dynhash_lookup (fp->ctf_link_inputs, one_name);
+      else
+	one_input = (ctf_link_input_t *) input;
+
+      if (!one_input || (!one_input->clin_arc && !one_input->clin_fp))
+	continue;
+
+      /* Short-circuit: if clin_fp is set, just use it.   */
+      if (one_input->clin_fp)
+	{
+	  parents_[walk - dedup_inputs] = walk - dedup_inputs;
+	  *walk = one_input->clin_fp;
+	  walk++;
+	  continue;
+	}
+
+      /* Get and insert the parent archive (if any), if this archive has
+	 multiple members.  We assume, as elsewhere, that the parent is named
+	 _CTF_SECTION.  */
+
+      if ((parent_fp = ctf_arc_open_by_name (one_input->clin_arc,
+					     _CTF_SECTION, &err)) == NULL)
+	{
+	  if (err != ECTF_NOMEMBNAM)
+	    {
+	      ctf_next_destroy (i);
+	      ctf_set_errno (fp, err);
+	      goto err;
+	    }
+	}
+      else
+	{
+	  *walk = parent_fp;
+	  parent_i = walk - dedup_inputs;
+	  walk++;
+	}
+
+      /* We disregard the input archive name: either it is the parent (which we
+	 already have), or we want to put everything into one TU sharing the
+	 cuname anyway (if this is a CU-mapped link), or this is the final phase
+	 of a relink with CU-mapping off (i.e. ld -r) in which case the cuname
+	 is correctly set regardless.  */
+      while ((one_fp = ctf_archive_next (one_input->clin_arc, &j, NULL,
+					 1, &err)) != NULL)
+	{
+	  if (one_fp->ctf_flags & LCTF_CHILD)
+	    {
+	      /* The contents of the parents array for elements not
+		 corresponding to children is undefined.  If there is no parent
+		 (itself a sign of a likely linker bug or corrupt input), we set
+		 it to itself.  */
+
+	      ctf_import (one_fp, parent_fp);
+	      if (parent_fp)
+		parents_[walk - dedup_inputs] = parent_i;
+	      else
+		parents_[walk - dedup_inputs] = walk - dedup_inputs;
+	    }
+	  *walk = one_fp;
+	  walk++;
+	}
+      if (err != ECTF_NEXT_END)
+	{
+	  ctf_next_destroy (i);
+	  goto iterr;
+	}
+    }
+  if (err != ECTF_NEXT_END)
+    goto iterr;
+
+  *parents = parents_;
+
+  return dedup_inputs;
+
+ oom:
+  err = ENOMEM;
+
+ iterr:
+  ctf_set_errno (fp, err);
+
+ err:
+  free (dedup_inputs);
+  free (parents_);
+  ctf_err_warn (fp, 0, "Error in deduplicating CTF link input allocation: %s",
+		ctf_errmsg (ctf_errno (fp)));
+  return NULL;
+}
+
+/* Close INPUTS that have already been linked, first the passed array, and then
+   that subset of the ctf_link_inputs archives they came from cited by the
+   CU_NAMES.  If CU_NAMES is not specified, close all the ctf_link_inputs in one
+   go, leaving it empty.  */
+static int
+ctf_link_deduplicating_close_inputs (ctf_file_t *fp, ctf_dynhash_t *cu_names,
+				     ctf_file_t **inputs, ssize_t ninputs)
+{
+  ctf_next_t *it = NULL;
+  void *name;
+  int err;
+  ssize_t i;
+
+  /* This is the inverse of ctf_link_deduplicating_open_inputs: so first, close
+     all the individual input dicts, opened by the archive iterator.  */
+  for (i = 0; i < ninputs; i++)
+    ctf_file_close (inputs[i]);
+
+  /* Now close the archives they are part of.  */
+  if (cu_names)
+    {
+      while ((err = ctf_dynhash_next (cu_names, &it, &name, NULL)) == 0)
+	{
+	  /* Remove the input from the linker inputs, if it exists, which also
+	     closes it.  */
+
+	  ctf_dynhash_remove (fp->ctf_link_inputs, (const char *) name);
+	}
+      if (err != ECTF_NEXT_END)
+	{
+	  ctf_err_warn (fp, 0, "Iteration error in deduplicating link input "
+			"freeing: %s", ctf_errmsg (err));
+	  ctf_set_errno (fp, err);
+	}
+    }
+  else
+    ctf_dynhash_empty (fp->ctf_link_inputs);
+
+  return 0;
+}
+
+/* Do a deduplicating link of all variables in the inputs.  */
+static int
+ctf_link_deduplicating_variables (ctf_file_t *fp, ctf_file_t **inputs,
+				  size_t ninputs, int cu_mapped)
+{
+  ctf_link_in_member_cb_arg_t arg;
+  size_t i;
+
+  arg.cu_mapped = cu_mapped;
+  arg.out_fp = fp;
+  arg.in_input_cu_file = 0;
+
+  for (i = 0; i < ninputs; i++)
+    {
+      arg.in_fp = inputs[i];
+      if (ctf_cuname (inputs[i]) != NULL)
+	arg.in_file_name = ctf_cuname (inputs[i]);
+      else
+	arg.in_file_name = "unnamed-CU";
+      arg.cu_name = arg.in_file_name;
+      if (ctf_variable_iter (arg.in_fp, ctf_link_one_variable, &arg) < 0)
+	return ctf_set_errno (fp, ctf_errno (arg.in_fp));
+
+      /* Outputs > 0 are per-CU.  */
+      arg.in_input_cu_file = 1;
+    }
+  return 0;
+}
+
+/* Do the per-CU part of a deduplicating link.  */
+static int
+ctf_link_deduplicating_per_cu (ctf_file_t *fp)
+{
+  ctf_next_t *i = NULL;
+  int err;
+  void *out_cu;
+  void *in_cus;
+
+  /* Links with a per-CU mapping in force get a first pass of deduplication,
+     dedupping the inputs for a given CU mapping into the output for that
+     mapping.  The outputs from this process get fed back into the final pass
+     that is carried out even for non-CU links.  */
+
+  while ((err = ctf_dynhash_next (fp->ctf_link_out_cu_mapping, &i, &out_cu,
+				  &in_cus)) == 0)
+    {
+      const char *out_name = (const char *) out_cu;
+      ctf_dynhash_t *in = (ctf_dynhash_t *) in_cus;
+      ctf_file_t *out = NULL;
+      ctf_file_t **inputs;
+      ctf_file_t **outputs;
+      ctf_archive_t *in_arc;
+      ssize_t ninputs;
+      ctf_link_input_t *only_input;
+      uint32_t noutputs;
+      uint32_t *parents;
+
+      if ((ninputs = ctf_link_deduplicating_count_inputs (fp, in, &only_input)) < 0)
+	goto err_open_inputs;
+
+      /* CU mapping with no inputs?  Skip.  */
+      if (ninputs == 0)
+	continue;
+
+      if (ninputs > -1)
+	{
+	  ctf_err_warn (fp, 0, "Too many inputs in deduplicating link: %li",
+			(long int) ninputs);
+	  goto err_open_inputs;
+	}
+
+      /* Short-circuit: a cu-mapped link with only one input archive with
+	 unconflicting contents is a do-nothing, and we can just leave the input
+	 in place: we do have to change the cuname, though, so we unwrap it,
+	 change the cuname, then stuff it back in the linker input again, via
+	 the clin_fp short-circuit member.  ctf_link_deduplicating_open_inputs
+	 will spot this member and jam it straight into the next link phase,
+	 ignoring the corresponding archive.  */
+      if (only_input && ninputs == 1)
+	{
+	  ctf_next_t *ai = NULL;
+	  int err;
+
+	  /* We can abuse an archive iterator to get the only member cheaply, no
+	     matter what its name.  */
+	  only_input->clin_fp = ctf_archive_next (only_input->clin_arc,
+						  &ai, NULL, 0, &err);
+	  if (!only_input->clin_fp)
+	    {
+	      ctf_err_warn (fp, 0, "Cannot open archive %s in CU-mapped CTF "
+			    "link: %s", only_input->clin_filename,
+			    ctf_errmsg (err));
+	      ctf_set_errno (fp, err);
+	      goto err_open_inputs;
+	    }
+	  ctf_next_destroy (ai);
+
+	  if (strcmp (only_input->clin_filename, out_name) != 0)
+	    {
+	      /* Renaming. We need to add a new input, then null out the
+		 clin_arc and clin_fp of the old one to stop it being
+		 auto-closed on removal.  The new input needs its cuname changed
+		 to out_name, which is doable only because the cuname is a
+		 dynamic property which can be changed even in readonly
+		 dicts. */
+
+	      ctf_cuname_set (only_input->clin_fp, out_name);
+	      if (ctf_link_add_ctf_internal (fp, only_input->clin_arc,
+					     only_input->clin_fp,
+					     out_name) < 0)
+		{
+		  ctf_err_warn (fp, 0, "Cannot add intermediate files "
+				"to link: %s", ctf_errmsg (ctf_errno (fp)));
+		  goto err_open_inputs;
+		}
+	      only_input->clin_arc = NULL;
+	      only_input->clin_fp = NULL;
+	      ctf_dynhash_remove (fp->ctf_link_inputs,
+				  only_input->clin_filename);
+	    }
+	  continue;
+	}
+
+      /* This is a real CU many-to-one mapping: we must dedup the inputs into
+	 a new output to be used in the final link phase.  */
+
+      if ((inputs = ctf_link_deduplicating_open_inputs (fp, in, ninputs,
+							&parents)) == NULL)
+	{
+	  ctf_next_destroy (i);
+	  goto err_inputs;
+	}
+
+      if ((out = ctf_create (&err)) == NULL)
+	{
+	  ctf_err_warn (fp, 0, "Cannot create per-CU CTF archive for %s: %s",
+		       out_name, ctf_errmsg (err));
+	  ctf_set_errno (fp, err);
+	  goto err_inputs;
+	}
+
+      /* Share the atoms table to reduce memory usage.  */
+      out->ctf_dedup_atoms = fp->ctf_dedup_atoms_alloc;
+
+      /* No ctf_imports at this stage: this per-CU dictionary has no parents.
+	 Parent/child deduplication happens in the link's final pass.  However,
+	 the cuname *is* important, as it is propagated into the final
+	 dictionary.  */
+      ctf_cuname_set (out, out_name);
+
+      if (ctf_dedup (out, inputs, ninputs, parents, 1) < 0)
+	{
+	  ctf_err_warn (fp, 0, "CU-mapped deduplication failed for %s: %s",
+		       out_name, ctf_errmsg (ctf_errno (out)));
+	  goto err_inputs;
+	}
+
+      if ((outputs = ctf_dedup_emit (out, inputs, ninputs, parents,
+				     &noutputs, 1)) == NULL)
+	{
+	  ctf_err_warn (fp, 0, "CU-mapped deduplicating link type emission "
+			"failed for %s: %s", out_name,
+			ctf_errmsg (ctf_errno (out)));
+	  goto err_inputs;
+	}
+      if (!ctf_assert (fp, noutputs == 1))
+	goto err_inputs_outputs;
+
+      if (!(fp->ctf_link_flags & CTF_LINK_OMIT_VARIABLES_SECTION)
+	  && ctf_link_deduplicating_variables (out, inputs, ninputs, 1) < 0)
+	{
+	  ctf_err_warn (fp, 0, "CU-mapped deduplicating link variable "
+			"emission failed for %s: %s", out_name,
+			ctf_errmsg (ctf_errno (out)));
+	  goto err_inputs_outputs;
+	}
+
+      if (ctf_link_deduplicating_close_inputs (fp, in, inputs, ninputs) < 0)
+	{
+	  free (inputs);
+	  free (parents);
+	  goto err_outputs;
+	}
+      free (inputs);
+      free (parents);
+
+      /* Splice any errors or warnings created during this link back into the
+	 dict that the caller knows about.  */
+      ctf_list_splice (&fp->ctf_errs_warnings, &outputs[0]->ctf_errs_warnings);
+
+      /* This output now becomes an input to the next link phase, with a name
+	 equal to the CU name.  We have to wrap it in an archive wrapper
+	 first.  */
+
+      if ((in_arc = ctf_new_archive_internal (0, 0, NULL, outputs[0], NULL,
+					      NULL, &err)) == NULL)
+	{
+	  ctf_set_errno (fp, err);
+	  goto err_outputs;
+	}
+
+      if (ctf_link_add_ctf_internal (fp, in_arc, NULL,
+				     ctf_cuname (outputs[0])) < 0)
+	{
+	  ctf_err_warn (fp, 0, "Cannot add intermediate files to link: %s",
+			ctf_errmsg (ctf_errno (fp)));
+	  goto err_outputs;
+	}
+
+      ctf_file_close (out);
+      free (outputs);
+      continue;
+
+    err_inputs_outputs:
+      ctf_list_splice (&fp->ctf_errs_warnings, &outputs[0]->ctf_errs_warnings);
+      ctf_file_close (outputs[0]);
+      free (outputs);
+    err_inputs:
+      ctf_link_deduplicating_close_inputs (fp, in, inputs, ninputs);
+      ctf_file_close (out);
+      free (inputs);
+      free (parents);
+    err_open_inputs:
+      ctf_next_destroy (i);
+      return -1;
+
+    err_outputs:
+      ctf_list_splice (&fp->ctf_errs_warnings, &outputs[0]->ctf_errs_warnings);
+      ctf_file_close (outputs[0]);
+      free (outputs);
+      ctf_next_destroy (i);
+      return -1;				/* Errno is set for us.  */
+    }
+  if (err != ECTF_NEXT_END)
+    {
+      ctf_err_warn (fp, 0, "Iteration error in CU-mapped deduplicating "
+		    "link: %s", ctf_errmsg (err));
+      return ctf_set_errno (fp, err);
+    }
+
+  return 0;
+}
+
+/* Do a deduplicating link using the ctf-dedup machinery.  */
+static void
+ctf_link_deduplicating (ctf_file_t *fp)
+{
+  size_t i;
+  ctf_file_t **inputs, **outputs = NULL;
+  ssize_t ninputs;
+  uint32_t noutputs;
+  uint32_t *parents;
+
+  if (ctf_dedup_atoms_init (fp) < 0)
+    {
+      ctf_err_warn (fp, 0, "%s allocating CTF dedup atoms table",
+		    ctf_errmsg (ctf_errno (fp)));
+      return;					/* Errno is set for us.  */
+    }
+
+  if (fp->ctf_link_out_cu_mapping
+      && (ctf_link_deduplicating_per_cu (fp) < 0))
+    return;					/* Errno is set for us.  */
+
+  if ((ninputs = ctf_link_deduplicating_count_inputs (fp, NULL, NULL)) < 0)
+    return;					/* Errno is set for us.  */
+
+  if ((inputs = ctf_link_deduplicating_open_inputs (fp, NULL, ninputs,
+						    &parents)) == NULL)
+    return;					/* Errno is set for us.  */
+
+  if (ninputs == 1 && ctf_cuname (inputs[0]) != NULL)
+    ctf_cuname_set (fp, ctf_cuname (inputs[0]));
+
+  if (ctf_dedup (fp, inputs, ninputs, parents, 0) < 0)
+    {
+      ctf_err_warn (fp, 0, "Deduplication failed for %s: %s",
+		    ctf_link_input_name (fp), ctf_errmsg (ctf_errno (fp)));
+      goto err;
+    }
+
+  if ((outputs = ctf_dedup_emit (fp, inputs, ninputs, parents, &noutputs,
+				 0)) == NULL)
+    {
+      ctf_err_warn (fp, 0, "Deduplicating link type emission failed "
+		    "for %s: %s", ctf_link_input_name (fp),
+		    ctf_errmsg (ctf_errno (fp)));
+      goto err;
+    }
+
+  if (!ctf_assert (fp, outputs[0] == fp))
+    goto err;
+
+  for (i = 0; i < noutputs; i++)
+    {
+      char *dynname;
+
+      /* We already have access to this one.  Close the duplicate.  */
+      if (i == 0)
+	{
+	  ctf_file_close (outputs[0]);
+	  continue;
+	}
+
+      if ((dynname = strdup (ctf_cuname (outputs[i]))) == NULL)
+	goto oom_one_output;
+
+      if (ctf_dynhash_insert (fp->ctf_link_outputs, dynname, outputs[i]) < 0)
+	goto oom_one_output;
+
+      continue;
+
+    oom_one_output:
+      ctf_err_warn (fp, 0, "Out of memory allocating link outputs");
+      ctf_set_errno (fp, ENOMEM);
+      free (dynname);
+
+      for (; i < noutputs; i++)
+	ctf_file_close (outputs[i]);
+      goto err;
+    }
+
+  if (!(fp->ctf_link_flags & CTF_LINK_OMIT_VARIABLES_SECTION)
+      && ctf_link_deduplicating_variables (fp, inputs, ninputs, 0) < 0)
+    {
+      ctf_err_warn (fp, 0, "Deduplicating link variable emission failed for "
+		    "%s: %s", ctf_link_input_name (fp),
+		    ctf_errmsg (ctf_errno (fp)));
+      for (i = 1; i < noutputs; i++)
+	ctf_file_close (outputs[i]);
+      goto err;
+    }
+
+  /* Now close all the inputs, including per-CU intermediates.  */
+
+  if (ctf_link_deduplicating_close_inputs (fp, NULL, inputs, ninputs) < 0)
+    return;					/* errno is set for us.  */
+
+  ninputs = 0;					/* Prevent double-close.  */
+  ctf_set_errno (fp, 0);
+
+  /* Fall through.  */
+
+ err:
+  for (i = 0; i < (size_t) ninputs; i++)
+    ctf_file_close (inputs[i]);
+  free (inputs);
+  free (parents);
+  free (outputs);
+  return;
+}
+
 /* Merge types and variable sections in all files added to the link
    together.  All the added files are closed.  */
 int
@@ -871,8 +1523,11 @@ ctf_link (ctf_file_t *fp, int flags)
 	}
     }
 
-  ctf_dynhash_iter (fp->ctf_link_inputs, ctf_link_one_input_archive,
-		    &arg);
+  if ((flags & CTF_LINK_NONDEDUP) || (getenv ("LD_NO_CTF_DEDUP")))
+    ctf_dynhash_iter (fp->ctf_link_inputs, ctf_link_one_input_archive,
+		      &arg);
+  else
+    ctf_link_deduplicating (fp);
 
   /* Discard the now-unnecessary mapping table data from all the outputs.  */
   if (fp->ctf_link_type_mapping)
diff --git a/libctf/ctf-util.c b/libctf/ctf-util.c
index c113dfc5596..a0338f4742a 100644
--- a/libctf/ctf-util.c
+++ b/libctf/ctf-util.c
@@ -88,6 +88,26 @@ ctf_list_empty_p (ctf_list_t *lp)
   return (lp->l_next == NULL && lp->l_prev == NULL);
 }
 
+/* Splice one entire list onto the end of another one.  The existing list is
+   emptied.  */
+
+void
+ctf_list_splice (ctf_list_t *lp, ctf_list_t *append)
+{
+  if (ctf_list_empty_p (append))
+    return;
+
+  if (lp->l_prev != NULL)
+    lp->l_prev->l_next = append->l_next;
+  else
+    lp->l_next = append->l_next;
+
+  append->l_next->l_prev = lp->l_prev;
+  lp->l_prev = append->l_prev;
+  append->l_next = NULL;
+  append->l_prev = NULL;
+}
+
 /* Convert a 32-bit ELF symbol into Elf64 and return a pointer to it.  */
 
 Elf64_Sym *
-- 
2.27.0.247.g3dff7de930

[PATCH 53/59] binutils: objdump: ctf: drop incorrect linefeeds

[Nick Alcock]

The CTF objdumping code is adding linefeeds in calls to non_fatal, which
is wrong and looks ugly.

binutils/
	* objdump.c (dump_ctf_archive_member): Remove linefeeds.
	(dump_ctf): Likewise.
---
 binutils/objdump.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/binutils/objdump.c b/binutils/objdump.c
index 8d4b1b6ba94..978d86cb1fd 100644
--- a/binutils/objdump.c
+++ b/binutils/objdump.c
@@ -4115,7 +4115,7 @@ dump_ctf_archive_member (ctf_file_t *ctf, const char *name, void *arg)
 
       if (ctf_errno (ctf))
 	{
-	  non_fatal (_("Iteration failed: %s, %s\n"), *thing,
+	  non_fatal (_("Iteration failed: %s, %s"), *thing,
 		   ctf_errmsg (ctf_errno (ctf)));
 	  break;
 	}
@@ -4162,7 +4162,7 @@ dump_ctf (bfd *abfd, const char *sect_name, const char *parent_name)
   ctfsect = make_ctfsect (sect_name, ctfdata, ctfsize);
   if ((ctfa = ctf_bfdopen_ctfsect (abfd, &ctfsect, &err)) == NULL)
     {
-      non_fatal (_("CTF open failure: %s\n"), ctf_errmsg (err));
+      non_fatal (_("CTF open failure: %s"), ctf_errmsg (err));
       bfd_fatal (bfd_get_filename (abfd));
     }
 
@@ -4171,7 +4171,7 @@ dump_ctf (bfd *abfd, const char *sect_name, const char *parent_name)
       ctfsect = make_ctfsect (parent_name, parentdata, parentsize);
       if ((parenta = ctf_bfdopen_ctfsect (abfd, &ctfsect, &err)) == NULL)
 	{
-	  non_fatal (_("CTF open failure: %s\n"), ctf_errmsg (err));
+	  non_fatal (_("CTF open failure: %s"), ctf_errmsg (err));
 	  bfd_fatal (bfd_get_filename (abfd));
 	}
 
@@ -4185,7 +4185,7 @@ dump_ctf (bfd *abfd, const char *sect_name, const char *parent_name)
      put CTFs and their parents in archives together.)  */
   if ((parent = ctf_arc_open_by_name (lookparent, NULL, &err)) == NULL)
     {
-      non_fatal (_("CTF open failure: %s\n"), ctf_errmsg (err));
+      non_fatal (_("CTF open failure: %s"), ctf_errmsg (err));
       bfd_fatal (bfd_get_filename (abfd));
     }
 
-- 
2.27.0.247.g3dff7de930

[PATCH 54/59] ld: Reformat CTF errors into warnings.

[Nick Alcock]

From: Egeyar Bagcioglu <egeyar.bagcioglu@oracle.com>

ld/
	* ldlang.c (lang_merge_ctf): Turn errors into warnings.
	Fix a comment typo.
	(lang_write_ctf): Turn an error into a warning.
	(ldlang_open_ctf): Reformat warnings. Fix printing file names.

Reviewed-by: Nick Alcock <nick.alcock@oracle.com>
---
 ld/ldlang.c | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/ld/ldlang.c b/ld/ldlang.c
index 10bdc86f8d7..ea947a8cd3c 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -3694,8 +3694,8 @@ ldlang_open_ctf (void)
       if ((file->the_ctf = ctf_bfdopen (file->the_bfd, &err)) == NULL)
 	{
 	  if (err != ECTF_NOCTFDATA)
-	    einfo (_("%P: warning: CTF section in `%pI' not loaded: "
-		     "its types will be discarded: `%s'\n"), file,
+	    einfo (_("%P: warning: CTF section in %pB not loaded; "
+		     "its types will be discarded: `%s'\n"), file->the_bfd,
 		     ctf_errmsg (err));
 	  continue;
 	}
@@ -3781,11 +3781,11 @@ lang_merge_ctf (void)
       if (!file->the_ctf)
 	continue;
 
-      /* Takes ownership of file->u.the_ctfa.  */
+      /* Takes ownership of file->the_ctf.  */
       if (ctf_link_add_ctf (ctf_output, file->the_ctf, file->filename) < 0)
 	{
-	  einfo (_("%F%P: cannot link with CTF in %pB: %s\n"), file->the_bfd,
-		 ctf_errmsg (ctf_errno (ctf_output)));
+	  einfo (_("%P: warning: CTF section in %pB cannot be linked: `%s'\n"),
+		 file->the_bfd, ctf_errmsg (ctf_errno (ctf_output)));
 	  ctf_close (file->the_ctf);
 	  file->the_ctf = NULL;
 	  continue;
@@ -3794,7 +3794,8 @@ lang_merge_ctf (void)
 
   if (ctf_link (ctf_output, CTF_LINK_SHARE_UNCONFLICTED) < 0)
     {
-      einfo (_("%F%P: CTF linking failed; output will have no CTF section: %s\n"),
+      einfo (_("%P: warning: CTF linking failed; "
+	       "output will have no CTF section: `%s'\n"),
 	     ctf_errmsg (ctf_errno (ctf_output)));
       if (output_sect)
 	{
@@ -3851,8 +3852,9 @@ lang_write_ctf (int late)
 
       if (!output_sect->contents)
 	{
-	  einfo (_("%F%P: CTF section emission failed; output will have no "
-		   "CTF section: %s\n"), ctf_errmsg (ctf_errno (ctf_output)));
+	  einfo (_("%P: warning: CTF section emission failed; "
+		   "output will have no CTF section: `%s'\n"),
+		 ctf_errmsg (ctf_errno (ctf_output)));
 	  output_sect->size = 0;
 	  output_sect->flags |= SEC_EXCLUDE;
 	}
@@ -3891,8 +3893,8 @@ ldlang_open_ctf (void)
 
       if ((sect = bfd_get_section_by_name (file->the_bfd, ".ctf")) != NULL)
 	{
-	    einfo (_("%P: warning: CTF section in `%pI' not linkable: "
-		     "%P was built without support for CTF\n"), file);
+	    einfo (_("%P: warning: CTF section in %pB not linkable: "
+		     "%P was built without support for CTF\n"), file->the_bfd);
 	    sect->size = 0;
 	    sect->flags |= SEC_EXCLUDE;
 	}
-- 
2.27.0.247.g3dff7de930

[PATCH 55/59] ld: new options --ctf-variables and --ctf-share-types

[Nick Alcock]

libctf recently changed to make it possible to not emit the CTF
variables section.  Make this the default for ld: the variables section
is a simple name -> type mapping, and the names can be quite voluminous.
Nothing in the variables section appears in the symbol table, by
definition, so GDB cannot make use of them: special-purpose projects
that implement their own analogues of symbol table lookup can do so, but
they'll need to tell the linker to emit the variables section after all.

The new --ctf-variables option does this.

The --ctf-share-types option (valid values "share-duplicated" and
"share-unconflicted") allow the caller to specify the CTF link mode.
Most users will want share-duplicated, since it allows for more
convenient debugging: but very large projects composed of many decoupled
components may want to use share-unconflicted mode, which places types
that appear in only one TU into per-TU dicts.  (They may also want to
relink the CTF using the ctf_link API and cu-mapping, to make their
"components" larger than a single TU.  Right now the linker does not
expose the CU-mapping machinery.  Perhaps it should in future to make
this use case easier.)

For now, giving the linker the ability to emit share-duplicated CTF lets
us add testcases for that mode to the testsuite.

ld/
	* ldlex.h (option_values) <OPTION_CTF_VARIABLES,
	OPTION_NO_CTF_VARIABLES, OPTION_CTF_SHARE_TYPES>: New.
	* ld.h (ld_config_type) <ctf_variables, ctf_share_duplicated>:
	New fields.
	* ldlang.c (lang_merge_ctf): Use them.
	* lexsup.c (ld_options): Add ctf-variables, no-ctf-variables,
	ctf-share-types.
	(parse_args) <OPTION_CTF_VARIABLES, OPTION_NO_CTF_VARIABLES,
	OPTION_CTF_SHARE_TYPES>: New cases.
	* ld.texi: Document new options.
	* NEWS: Likewise.
---
 ld/NEWS     | 10 ++++++++++
 ld/ld.h     |  8 ++++++++
 ld/ld.texi  | 34 ++++++++++++++++++++++++++++++++++
 ld/ldlang.c | 10 +++++++++-
 ld/ldlex.h  |  3 +++
 ld/lexsup.c | 29 +++++++++++++++++++++++++++++
 6 files changed, 93 insertions(+), 1 deletion(-)

diff --git a/ld/NEWS b/ld/NEWS
index b236e588c6f..f7241a87c1d 100644
--- a/ld/NEWS
+++ b/ld/NEWS
@@ -1,5 +1,15 @@
 -*- text -*-
 
+* The linker now deduplicates the types in .ctf sections.  The new
+  command-line option --ctf-share-types describes how to do this:
+  its default value, share-unconflicted, produces the most compact
+  output.
+
+* The linker now omits the "variable section" from .ctf sections by
+  default, saving space.  This is almost certainly what you want
+  unless you are working on a project that has its own analogue
+  of symbol tables that are not reflected in the ELF symtabs.
+
 * X86 NaCl target support is removed.
 
 * Add ELF linker command-line options, --export-dynamic-symbol and
diff --git a/ld/ld.h b/ld/ld.h
index 1790dc81a66..7f7d71672e6 100644
--- a/ld/ld.h
+++ b/ld/ld.h
@@ -302,6 +302,14 @@ typedef struct
 
   /* If set, print discarded sections in map file output.  */
   bfd_boolean print_map_discarded;
+
+  /* If set, emit the names and types of statically-linked variables
+     into the CTF.  */
+  bfd_boolean ctf_variables;
+
+  /* If set, share only duplicated types in CTF, rather than sharing
+     all types that are not in conflict.  */
+  bfd_boolean ctf_share_duplicated;
 } ld_config_type;
 
 extern ld_config_type config;
diff --git a/ld/ld.texi b/ld/ld.texi
index 40b042de9b9..2a740706e0c 100644
--- a/ld/ld.texi
+++ b/ld/ld.texi
@@ -1612,6 +1612,40 @@ definition.  If the symbol is defined as a common value then any files
 where this happens appear next.  Finally any files that reference the
 symbol are listed.
 
+@cindex ctf variables
+@kindex --ctf-variables
+@kindex --no-ctf-variables
+@item --ctf-variables
+@item --no-ctf-variables
+The CTF debuginfo format supports a section which encodes the names and
+types of variables found in the program which do not appear in any symbol
+table. These variables clearly cannot be looked up by address by
+conventional debuggers, so the space used for their types and names is
+usually wasted: the types are usually small but the names are often not.
+@option{--ctf-variables} causes the generation of such a section.
+The default behaviour can be restored with @option{--no-ctf-variables}.
+
+@cindex ctf type sharing
+@kindex --ctf-share-types
+@item --ctf-share-types=@var{method}
+Adjust the method used to share types between translation units in CTF.
+
+@table @samp
+@item share-unconflicted
+Put all types that do not have ambiguous definitions into the shared dictionary,
+where debuggers can easily access them, even if they only occur in one
+translation unit.  This is the default.
+
+@item share-duplicated
+Put only types that occur in multiple translation units into the shared
+dictionary: types with only one definition go into per-translation-unit
+dictionaries.  Types with ambiguous definitions in multiple translation units
+always go into per-translation-unit dictionaries.  This tends to make the CTF
+larger, but may reduce the amount of CTF in the shared dictionary.  For very
+large projects this may speed up opening the CTF and save memory in the CTF
+consumer at runtime.
+@end table
+
 @cindex common allocation
 @kindex --no-define-common
 @item --no-define-common
diff --git a/ld/ldlang.c b/ld/ldlang.c
index ea947a8cd3c..da89838ed3d 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -3756,6 +3756,7 @@ static void
 lang_merge_ctf (void)
 {
   asection *output_sect;
+  int flags = 0;
 
   if (!ctf_output)
     return;
@@ -3792,7 +3793,14 @@ lang_merge_ctf (void)
 	}
     }
 
-  if (ctf_link (ctf_output, CTF_LINK_SHARE_UNCONFLICTED) < 0)
+  if (!config.ctf_share_duplicated)
+    flags = CTF_LINK_SHARE_UNCONFLICTED;
+  else
+    flags = CTF_LINK_SHARE_DUPLICATED;
+  if (!config.ctf_variables)
+    flags |= CTF_LINK_OMIT_VARIABLES_SECTION;
+
+  if (ctf_link (ctf_output, flags) < 0)
     {
       einfo (_("%P: warning: CTF linking failed; "
 	       "output will have no CTF section: `%s'\n"),
diff --git a/ld/ldlex.h b/ld/ldlex.h
index 5ea083ebeb3..6eac75cef53 100644
--- a/ld/ldlex.h
+++ b/ld/ldlex.h
@@ -155,6 +155,9 @@ enum option_values
   OPTION_NON_CONTIGUOUS_REGIONS,
   OPTION_NON_CONTIGUOUS_REGIONS_WARNINGS,
   OPTION_DEPENDENCY_FILE,
+  OPTION_CTF_VARIABLES,
+  OPTION_NO_CTF_VARIABLES,
+  OPTION_CTF_SHARE_TYPES,
 };
 
 /* The initial parser states.  */
diff --git a/ld/lexsup.c b/ld/lexsup.c
index 58c6c078325..d8b06736ed6 100644
--- a/ld/lexsup.c
+++ b/ld/lexsup.c
@@ -572,6 +572,18 @@ static const struct ld_option ld_options[] =
   { {"no-print-map-discarded", no_argument, NULL, OPTION_NO_PRINT_MAP_DISCARDED},
     '\0', NULL, N_("Do not show discarded sections in map file output"),
     TWO_DASHES },
+  { {"ctf-variables", no_argument, NULL, OPTION_CTF_VARIABLES},
+    '\0', NULL, N_("Emit names and types of static variables in CTF"),
+    TWO_DASHES },
+  { {"no-ctf-variables", no_argument, NULL, OPTION_NO_CTF_VARIABLES},
+    '\0', NULL, N_("Do not emit names and types of static variables in CTF"),
+    TWO_DASHES },
+  { {"ctf-share-types=<method>", required_argument, NULL,
+     OPTION_CTF_SHARE_TYPES},
+    '\0', NULL, N_("How to share CTF types between translation units.\n"
+		   "                                <method> is: share-unconflicted (default),\n"
+		   "                                             share-duplicated"),
+    TWO_DASHES },
 };
 
 #define OPTION_COUNT ARRAY_SIZE (ld_options)
@@ -1637,6 +1649,23 @@ parse_args (unsigned argc, char **argv)
 	case OPTION_DEPENDENCY_FILE:
 	  config.dependency_file = optarg;
 	  break;
+
+	case OPTION_CTF_VARIABLES:
+	  config.ctf_variables = TRUE;
+	  break;
+
+	case OPTION_NO_CTF_VARIABLES:
+	  config.ctf_variables = FALSE;
+	  break;
+
+	case OPTION_CTF_SHARE_TYPES:
+	  if (strcmp (optarg, "share-unconflicted") == 0)
+	    config.ctf_share_duplicated = FALSE;
+	  else if (strcmp (optarg, "share-duplicated") == 0)
+	    config.ctf_share_duplicated = TRUE;
+	  else
+	    einfo (_("%F%P: bad --ctf-share-types option: %s\n"), optarg);
+	  break;
 	}
     }
 
-- 
2.27.0.247.g3dff7de930

[PATCH 56/59] binutils, testsuite: allow compilation before doing run_dump_test

[Nick Alcock]

The CTF assembler emitted by GCC has architecture-dependent pseudos in
it, and is (obviously) tightly tied to a particular set of C source
files with specific types in them.  The CTF tests do run_dump_test on
some candidate input, link it using the run_dump_test ld machinery, and
compare objdump --ctf output.  To avoid skew, we'd like to be able
to easily regenerate the .s being scanned so that the .c doesn't get
out of sync with it, but since GCC emits arch-dependent pseudos, we
are forced to hand-hack the output every time (quite severely on some
arches, like x86-32 and -64, where every single pseudo used is not only
arch-dependent but undocumented).

To avoid this, teach run_dump_test how to optionally compile things
given new, optional additional flags passed in in the cc option.
Only sources with the .c suffix are compiled, so there is no effect on
any existing tests.  The .s files go into the tmpdir, from which
existing run_dump_test code picks them up as usual.

binutils/
	* testsuite/lib/binutils-common.exp (run_dump_test): Add 'cc'
	option.
---
 binutils/testsuite/lib/binutils-common.exp | 58 ++++++++++++++++++++--
 1 file changed, 53 insertions(+), 5 deletions(-)

diff --git a/binutils/testsuite/lib/binutils-common.exp b/binutils/testsuite/lib/binutils-common.exp
index 491cd8db09e..10b243ee064 100644
--- a/binutils/testsuite/lib/binutils-common.exp
+++ b/binutils/testsuite/lib/binutils-common.exp
@@ -561,6 +561,7 @@ if ![string length [info proc prune_warnings]] {
 # run_dump_test FILE (optional:) EXTRA_OPTIONS
 #
 # Assemble a .s file, then run some utility on it and check the output.
+# Optionally generate the .s file first by running the compiler.
 #
 # There should be an assembly language file named FILE.s in the test
 # suite directory, and a pattern file called FILE.d.  run_dump_test
@@ -619,6 +620,11 @@ if ![string length [info proc prune_warnings]] {
 #   ld_after_inputfiles: FLAGS
 #	Similar to "ld", but put FLAGS after all input files.
 #
+#   cc: FLAGS
+#       Run the compiler with FLAGS (to which -S is added) to generate assembler
+#       source first.  source: must be provided and should consist of .c files.
+#       Source-specific CC flags are not supported.
+#
 #   objcopy_objects: FLAGS
 #	Run objcopy with the specified flags after assembling any source
 #	that has the special marker RUN_OBJCOPY in the source specific
@@ -745,8 +751,8 @@ if ![string length [info proc prune_warnings]] {
 # regexps in FILE.d.
 #
 proc run_dump_test { name {extra_options {}} } {
-    global ADDR2LINE ADDR2LINEFLAGS AS ASFLAGS ELFEDIT ELFEDITFLAGS LD LDFLAGS
-    global NM NMFLAGS OBJCOPY OBJCOPYFLAGS OBJDUMP OBJDUMPFLAGS
+    global ADDR2LINE ADDR2LINEFLAGS AS ASFLAGS CC CFLAGS ELFEDIT ELFEDITFLAGS
+    global LD LDFLAGS NM NMFLAGS OBJCOPY OBJCOPYFLAGS OBJDUMP OBJDUMPFLAGS
     global READELF READELFFLAGS STRIP STRIPFLAGS
     global copyfile env runtests srcdir subdir verbose
 
@@ -780,6 +786,7 @@ proc run_dump_test { name {extra_options {}} } {
     set opts(as) {}
     set as_final_flags {}
     set as_additional_flags {}
+    set opts(cc) {}
     set opts(dump) {}
     set opts(elfedit) {}
     set opts(error) {}
@@ -823,10 +830,10 @@ proc run_dump_test { name {extra_options {}} } {
 	    return
 	}
 
-	# Allow more substitutions, including tcl functions, for as and ld.
-	# Not done in general because extra quoting is needed for glob
+	# Allow more substitutions, including tcl functions, for as, ld,
+	# and cc.  Not done in general because extra quoting is needed for glob
 	# args used for example in binutils-all/remove-relocs-04.d.
-	if { $opt_name == "as" || $opt_name == "ld" } {
+	if { $opt_name == "as" || $opt_name == "ld" || $opt_name == "cc" } {
 	    set opt_val [subst $opt_val]
 	} else {
 	    # Just substitute $srcdir and $subdir
@@ -1032,6 +1039,47 @@ proc run_dump_test { name {extra_options {}} } {
 	}
     }
 
+    # Possibly compile some of the inputs, and build up a replacement
+    # for opts(source) with the output .s names substituted in as we go.
+    # Set the .s names from the objfile_names to take advantage of the
+    # uniquification that happened earlier.
+    if { $opts(cc) != ""} {
+	set cmdret 0
+	set new_source ""
+
+	foreach cfile $opts(source) ofile $objfile_names {
+	    if { [file extension $cfile] != ".c" } {
+		lappend new_source "$cfile"
+		continue
+	    }
+
+	    if { ! [string match "./*" $cfile] } {
+		set cfile "$srcdir/$subdir/$cfile"
+	    }
+	    # ofile is never absolute, so this always works to protect sfile
+	    # from later absolutization.
+	    set sfile "./[file rootname $ofile].s"
+	    set cmd "$CC $CFLAGS -S $opts(cc) -o $sfile $cfile"
+	    send_log "$cmd\n"
+	    set cmdret [remote_exec host [concat sh -c [list "$cmd 2>&1"]] "" "/dev/null" "dump.tmp"]
+	    remote_upload host "dump.tmp"
+	    set comp_output [prune_warnings [file_contents "dump.tmp"]]
+	    remote_file host delete "dump.tmp"
+	    remote_file build delete "dump.tmp"
+	    lappend new_source "$sfile"
+	    set cmdret [lindex $cmdret 0]
+
+	    regsub "\n$" $comp_output "" comp_output
+	    if { $cmdret != 0} {
+		send_log "compilation of $cfile failed, exit status $cmdret with <$comp_output>"
+                # Should this be 'unresolved', or is that too silent?
+		fail $testname
+		return 0
+	    }
+	}
+	set opts(source) $new_source
+    }
+
     if { $opts(source) == "" } {
 	set sourcefiles [list ${file}.s]
 	set asflags [list ""]
-- 
2.27.0.247.g3dff7de930

[PATCH 57/59] ld: new CTF testsuite

[Nick Alcock]

From: Egeyar Bagcioglu <egeyar.bagcioglu@oracle.com>

Uses the new cc option to run_dump_test to compile most tests from C
code, ensuring that the types in the C code accurately describe what the
.d file is testing.

(Some tests, mostly those testing malformed CTF, run directly from .s,
or include both .s and .c.)

ld/
	* testsuite/ld-ctf/ctf.exp: New file.
	* testsuite/ld-ctf/A-2.c: New file.
	* testsuite/ld-ctf/A.c: New file.
	* testsuite/ld-ctf/B-2.c: New file.
	* testsuite/ld-ctf/B.c: New file.
	* testsuite/ld-ctf/C-2.c: New file.
	* testsuite/ld-ctf/C.c: New file.
	* testsuite/ld-ctf/array-char.c: New file.
	* testsuite/ld-ctf/array-int.c: New file.
	* testsuite/ld-ctf/array.d: New file.
	* testsuite/ld-ctf/child-float.c: New file.
	* testsuite/ld-ctf/child-int.c: New file.
	* testsuite/ld-ctf/conflicting-cycle-1.B-1.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-1.B-2.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-1.parent.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-2.A-1.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-2.A-2.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-2.parent.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-3.C-1.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-3.C-2.d: New file.
	* testsuite/ld-ctf/conflicting-cycle-3.parent.d: New file.
	* testsuite/ld-ctf/conflicting-enums.d: New file.
	* testsuite/ld-ctf/conflicting-typedefs.d: New file.
	* testsuite/ld-ctf/cross-tu-1.c: New file.
	* testsuite/ld-ctf/cross-tu-2.c: New file.
	* testsuite/ld-ctf/cross-tu-conflicting-2.c: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-1.c: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-2.c: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-3.c: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-4.c: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-conflicting.d: New file.
	* testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d: New file.
	* testsuite/ld-ctf/cross-tu-into-cycle.d: New file.
	* testsuite/ld-ctf/cross-tu-noncyclic.d: New file.
	* testsuite/ld-ctf/cycle-1.c: New file.
	* testsuite/ld-ctf/cycle-1.d: New file.
	* testsuite/ld-ctf/cycle-2.A.d: New file.
	* testsuite/ld-ctf/cycle-2.B.d: New file.
	* testsuite/ld-ctf/cycle-2.C.d: New file.
	* testsuite/ld-ctf/diag-ctf-version-0.d: New file.
	* testsuite/ld-ctf/diag-ctf-version-0.s: New file.
	* testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d: New file.
	* testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s: New file.
	* testsuite/ld-ctf/diag-ctf-version-f.d: New file.
	* testsuite/ld-ctf/diag-ctf-version-f.s: New file.
	* testsuite/ld-ctf/diag-cttname-invalid.d: New file.
	* testsuite/ld-ctf/diag-cttname-invalid.s: New file.
	* testsuite/ld-ctf/diag-cttname-null.d: New file.
	* testsuite/ld-ctf/diag-cttname-null.s: New file.
	* testsuite/ld-ctf/diag-cuname.d: New file.
	* testsuite/ld-ctf/diag-cuname.s: New file.
	* testsuite/ld-ctf/diag-decompression-failure.d: New file.
	* testsuite/ld-ctf/diag-decompression-failure.s: New file.
	* testsuite/ld-ctf/diag-parlabel.d: New file.
	* testsuite/ld-ctf/diag-parlabel.s: New file.
	* testsuite/ld-ctf/diag-parname.d: New file.
	* testsuite/ld-ctf/diag-parname.s: New file.
	* testsuite/ld-ctf/diag-unsupported-flag.d: New file.
	* testsuite/ld-ctf/diag-unsupported-flag.s: New file.
	* testsuite/ld-ctf/diag-wrong-magic-number-mixed.d: New file.
	* testsuite/ld-ctf/diag-wrong-magic-number.d: New file.
	* testsuite/ld-ctf/diag-wrong-magic-number.s: New file.
	* testsuite/ld-ctf/enum-2.c: New file.
	* testsuite/ld-ctf/enum.c: New file.
	* testsuite/ld-ctf/function.c: New file.
	* testsuite/ld-ctf/function.d: New file.
	* testsuite/ld-ctf/slice.c: New file.
	* testsuite/ld-ctf/slice.d: New file.
	* testsuite/ld-ctf/super-sub-cycles.c: New file.
	* testsuite/ld-ctf/super-sub-cycles.d: New file.
	* testsuite/ld-ctf/typedef-int.c: New file.
	* testsuite/ld-ctf/typedef-long.c: New file.
	* testsuite/ld-ctf/union-1.c: New file.
---
 ld/testsuite/ld-ctf/A-2.c                     |  6 ++
 ld/testsuite/ld-ctf/A.c                       |  5 ++
 ld/testsuite/ld-ctf/B-2.c                     |  5 ++
 ld/testsuite/ld-ctf/B.c                       |  4 ++
 ld/testsuite/ld-ctf/C-2.c                     |  5 ++
 ld/testsuite/ld-ctf/C.c                       |  5 ++
 ld/testsuite/ld-ctf/array-char.c              |  2 +
 ld/testsuite/ld-ctf/array-int.c               |  1 +
 ld/testsuite/ld-ctf/array.d                   | 34 ++++++++++
 ld/testsuite/ld-ctf/child-float.c             |  4 ++
 ld/testsuite/ld-ctf/child-int.c               |  4 ++
 ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d | 40 ++++++++++++
 ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d | 41 ++++++++++++
 .../ld-ctf/conflicting-cycle-1.parent.d       | 38 +++++++++++
 ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d | 40 ++++++++++++
 ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d | 41 ++++++++++++
 .../ld-ctf/conflicting-cycle-2.parent.d       | 40 ++++++++++++
 ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d | 39 +++++++++++
 ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d | 40 ++++++++++++
 .../ld-ctf/conflicting-cycle-3.parent.d       | 37 +++++++++++
 ld/testsuite/ld-ctf/conflicting-enums.d       | 35 ++++++++++
 ld/testsuite/ld-ctf/conflicting-typedefs.d    | 33 ++++++++++
 ld/testsuite/ld-ctf/cross-tu-1.c              | 12 ++++
 ld/testsuite/ld-ctf/cross-tu-2.c              |  8 +++
 ld/testsuite/ld-ctf/cross-tu-conflicting-2.c  |  8 +++
 ld/testsuite/ld-ctf/cross-tu-cyclic-1.c       | 14 ++++
 ld/testsuite/ld-ctf/cross-tu-cyclic-2.c       | 16 +++++
 ld/testsuite/ld-ctf/cross-tu-cyclic-3.c       |  3 +
 ld/testsuite/ld-ctf/cross-tu-cyclic-4.c       |  4 ++
 .../ld-ctf/cross-tu-cyclic-conflicting.d      | 57 +++++++++++++++++
 .../ld-ctf/cross-tu-cyclic-nonconflicting.d   | 50 +++++++++++++++
 ld/testsuite/ld-ctf/cross-tu-into-cycle.d     | 64 +++++++++++++++++++
 ld/testsuite/ld-ctf/cross-tu-noncyclic.d      | 46 +++++++++++++
 ld/testsuite/ld-ctf/ctf.exp                   | 26 ++++++++
 ld/testsuite/ld-ctf/cycle-1.c                 |  7 ++
 ld/testsuite/ld-ctf/cycle-1.d                 | 36 +++++++++++
 ld/testsuite/ld-ctf/cycle-2.A.d               | 40 ++++++++++++
 ld/testsuite/ld-ctf/cycle-2.B.d               | 40 ++++++++++++
 ld/testsuite/ld-ctf/cycle-2.C.d               | 40 ++++++++++++
 ld/testsuite/ld-ctf/diag-ctf-version-0.d      |  5 ++
 ld/testsuite/ld-ctf/diag-ctf-version-0.s      | 44 +++++++++++++
 .../diag-ctf-version-2-unsupported-feature.d  |  5 ++
 .../diag-ctf-version-2-unsupported-feature.s  | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-ctf-version-f.d      |  5 ++
 ld/testsuite/ld-ctf/diag-ctf-version-f.s      | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-cttname-invalid.d    |  5 ++
 ld/testsuite/ld-ctf/diag-cttname-invalid.s    | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-cttname-null.d       | 24 +++++++
 ld/testsuite/ld-ctf/diag-cttname-null.s       | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-cuname.d             | 39 +++++++++++
 ld/testsuite/ld-ctf/diag-cuname.s             | 44 +++++++++++++
 .../ld-ctf/diag-decompression-failure.d       |  5 ++
 .../ld-ctf/diag-decompression-failure.s       | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-parlabel.d           | 39 +++++++++++
 ld/testsuite/ld-ctf/diag-parlabel.s           | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-parname.d            |  5 ++
 ld/testsuite/ld-ctf/diag-parname.s            | 44 +++++++++++++
 ld/testsuite/ld-ctf/diag-unsupported-flag.d   |  5 ++
 ld/testsuite/ld-ctf/diag-unsupported-flag.s   | 44 +++++++++++++
 .../ld-ctf/diag-wrong-magic-number-mixed.d    | 39 +++++++++++
 ld/testsuite/ld-ctf/diag-wrong-magic-number.d |  5 ++
 ld/testsuite/ld-ctf/diag-wrong-magic-number.s | 44 +++++++++++++
 ld/testsuite/ld-ctf/enum-2.c                  |  3 +
 ld/testsuite/ld-ctf/enum.c                    |  3 +
 ld/testsuite/ld-ctf/function.c                |  3 +
 ld/testsuite/ld-ctf/function.d                | 23 +++++++
 ld/testsuite/ld-ctf/slice.c                   |  6 ++
 ld/testsuite/ld-ctf/slice.d                   | 30 +++++++++
 ld/testsuite/ld-ctf/super-sub-cycles.c        | 10 +++
 ld/testsuite/ld-ctf/super-sub-cycles.d        | 34 ++++++++++
 ld/testsuite/ld-ctf/typedef-int.c             |  3 +
 ld/testsuite/ld-ctf/typedef-long.c            |  3 +
 ld/testsuite/ld-ctf/union-1.c                 |  4 ++
 73 files changed, 1757 insertions(+)
 create mode 100644 ld/testsuite/ld-ctf/A-2.c
 create mode 100644 ld/testsuite/ld-ctf/A.c
 create mode 100644 ld/testsuite/ld-ctf/B-2.c
 create mode 100644 ld/testsuite/ld-ctf/B.c
 create mode 100644 ld/testsuite/ld-ctf/C-2.c
 create mode 100644 ld/testsuite/ld-ctf/C.c
 create mode 100644 ld/testsuite/ld-ctf/array-char.c
 create mode 100644 ld/testsuite/ld-ctf/array-int.c
 create mode 100644 ld/testsuite/ld-ctf/array.d
 create mode 100644 ld/testsuite/ld-ctf/child-float.c
 create mode 100644 ld/testsuite/ld-ctf/child-int.c
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-1.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-2.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-cycle-3.parent.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-enums.d
 create mode 100644 ld/testsuite/ld-ctf/conflicting-typedefs.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-1.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-conflicting-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-1.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-2.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-3.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-4.c
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-conflicting.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-into-cycle.d
 create mode 100644 ld/testsuite/ld-ctf/cross-tu-noncyclic.d
 create mode 100644 ld/testsuite/ld-ctf/ctf.exp
 create mode 100644 ld/testsuite/ld-ctf/cycle-1.c
 create mode 100644 ld/testsuite/ld-ctf/cycle-1.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.A.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.B.d
 create mode 100644 ld/testsuite/ld-ctf/cycle-2.C.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-0.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-0.s
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-f.d
 create mode 100644 ld/testsuite/ld-ctf/diag-ctf-version-f.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-invalid.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-invalid.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-null.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cttname-null.s
 create mode 100644 ld/testsuite/ld-ctf/diag-cuname.d
 create mode 100644 ld/testsuite/ld-ctf/diag-cuname.s
 create mode 100644 ld/testsuite/ld-ctf/diag-decompression-failure.d
 create mode 100644 ld/testsuite/ld-ctf/diag-decompression-failure.s
 create mode 100644 ld/testsuite/ld-ctf/diag-parlabel.d
 create mode 100644 ld/testsuite/ld-ctf/diag-parlabel.s
 create mode 100644 ld/testsuite/ld-ctf/diag-parname.d
 create mode 100644 ld/testsuite/ld-ctf/diag-parname.s
 create mode 100644 ld/testsuite/ld-ctf/diag-unsupported-flag.d
 create mode 100644 ld/testsuite/ld-ctf/diag-unsupported-flag.s
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number-mixed.d
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number.d
 create mode 100644 ld/testsuite/ld-ctf/diag-wrong-magic-number.s
 create mode 100644 ld/testsuite/ld-ctf/enum-2.c
 create mode 100644 ld/testsuite/ld-ctf/enum.c
 create mode 100644 ld/testsuite/ld-ctf/function.c
 create mode 100644 ld/testsuite/ld-ctf/function.d
 create mode 100644 ld/testsuite/ld-ctf/slice.c
 create mode 100644 ld/testsuite/ld-ctf/slice.d
 create mode 100644 ld/testsuite/ld-ctf/super-sub-cycles.c
 create mode 100644 ld/testsuite/ld-ctf/super-sub-cycles.d
 create mode 100644 ld/testsuite/ld-ctf/typedef-int.c
 create mode 100644 ld/testsuite/ld-ctf/typedef-long.c
 create mode 100644 ld/testsuite/ld-ctf/union-1.c

diff --git a/ld/testsuite/ld-ctf/A-2.c b/ld/testsuite/ld-ctf/A-2.c
new file mode 100644
index 00000000000..1484983fe8a
--- /dev/null
+++ b/ld/testsuite/ld-ctf/A-2.c
@@ -0,0 +1,6 @@
+struct A {
+  struct B *b;
+  int wombat;
+};
+
+static struct A a __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/A.c b/ld/testsuite/ld-ctf/A.c
new file mode 100644
index 00000000000..f99691a5d3a
--- /dev/null
+++ b/ld/testsuite/ld-ctf/A.c
@@ -0,0 +1,5 @@
+struct A {
+  struct B *b;
+};
+
+static struct A a __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/B-2.c b/ld/testsuite/ld-ctf/B-2.c
new file mode 100644
index 00000000000..be01fec1a29
--- /dev/null
+++ b/ld/testsuite/ld-ctf/B-2.c
@@ -0,0 +1,5 @@
+struct B {
+  struct C *c;
+  int wombat;
+};
+static struct B b __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/B.c b/ld/testsuite/ld-ctf/B.c
new file mode 100644
index 00000000000..8a7f4c2d3c2
--- /dev/null
+++ b/ld/testsuite/ld-ctf/B.c
@@ -0,0 +1,4 @@
+struct B {
+  struct C *c;
+};
+static struct B b __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/C-2.c b/ld/testsuite/ld-ctf/C-2.c
new file mode 100644
index 00000000000..12363b59609
--- /dev/null
+++ b/ld/testsuite/ld-ctf/C-2.c
@@ -0,0 +1,5 @@
+struct C {
+  struct A *a;
+  int wombat;
+};
+static struct C c __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/C.c b/ld/testsuite/ld-ctf/C.c
new file mode 100644
index 00000000000..02635e3804c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/C.c
@@ -0,0 +1,5 @@
+struct C {
+  struct A *a;
+};
+static struct C c __attribute__((used));
+
diff --git a/ld/testsuite/ld-ctf/array-char.c b/ld/testsuite/ld-ctf/array-char.c
new file mode 100644
index 00000000000..0d1c71e02c7
--- /dev/null
+++ b/ld/testsuite/ld-ctf/array-char.c
@@ -0,0 +1,2 @@
+char * digits_names[10] = {"zero", "one", "two", "three", "four",
+			   "five", "six", "seven", "eight", "nine"};
diff --git a/ld/testsuite/ld-ctf/array-int.c b/ld/testsuite/ld-ctf/array-int.c
new file mode 100644
index 00000000000..ec6bed2e038
--- /dev/null
+++ b/ld/testsuite/ld-ctf/array-int.c
@@ -0,0 +1 @@
+int digits[10] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, };
diff --git a/ld/testsuite/ld-ctf/array.d b/ld/testsuite/ld-ctf/array.d
new file mode 100644
index 00000000000..f45d68cfda7
--- /dev/null
+++ b/ld/testsuite/ld-ctf/array.d
@@ -0,0 +1,34 @@
+#as:
+#source: array-char.c
+#source: array-int.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Arrays
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0xf \(0x10 bytes\)
+    Type section:	0x10 -- 0x6b \(0x5c bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    digits.*\[10] .*
+    digits.*\[10] .*
+
+  Types:
+#...
+     [0-9a-f]*: .*\[10\] .*
+#...
+     [0-9a-f]*: .*\[10\] .*
+#...
diff --git a/ld/testsuite/ld-ctf/child-float.c b/ld/testsuite/ld-ctf/child-float.c
new file mode 100644
index 00000000000..7366dc4c9db
--- /dev/null
+++ b/ld/testsuite/ld-ctf/child-float.c
@@ -0,0 +1,4 @@
+struct child_float {
+  float d;
+};
+static struct child_float *a_child_float __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/child-int.c b/ld/testsuite/ld-ctf/child-int.c
new file mode 100644
index 00000000000..3454ce9159c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/child-int.c
@@ -0,0 +1,4 @@
+struct child_int {
+  int i;
+};
+static struct child_int *a_child_int __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d b/ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d
new file mode 100644
index 00000000000..146986bc3d2
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-1.B-1.d
@@ -0,0 +1,40 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 1.B-1
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/B.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .ctf
+    Compilation unit name: .*/B.c
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0x1f \(0x18 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    b ->  80000001: struct B \(size 0x[0-9]*\)
+
+  Types:
+     8[0-9a-f]*: struct B .*
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct B \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct C \* c \(.*
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d b/ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d
new file mode 100644
index 00000000000..48c11a79635
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-1.B-2.d
@@ -0,0 +1,41 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 1.B-2
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/B-2.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .ctf
+    Compilation unit name: .*/B-2.c
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0x2b \(0x24 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    b ->  80000001: struct B \(.*
+
+  Types:
+     8[0-9a-f]*: struct B \(.*
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct B \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct C \* c \(.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 1\) int wombat:32 \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-1.parent.d b/ld/testsuite/ld-ctf/conflicting-cycle-1.parent.d
new file mode 100644
index 00000000000..7e0c824da5c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-1.parent.d
@@ -0,0 +1,38 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 1.parent
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x17 \(0x18 bytes\)
+    Type section:	0x18 -- 0xc3 \(0xac bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+#...
+  Function objects:
+
+  Variables:
+#...
+  Types:
+#...
+     [0-9a-f]*: struct B \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 9\) struct B \(.*
+#...
+CTF archive member: .*:
+#...
+CTF archive member: .*:
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d b/ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d
new file mode 100644
index 00000000000..8929a1f2516
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-2.A-1.d
@@ -0,0 +1,40 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 2.A-1
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/A.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .*
+    Compilation unit name: .*/A.c
+#...
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    a ->  80000001: struct A \(size 0x[0-9a-f]*\)
+
+  Types:
+     8[0-9a-f]*: struct A \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct A \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(aligned at 0x[0-9a-f]*\)
+
+  Strings:
+    0: 
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d b/ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d
new file mode 100644
index 00000000000..f65382c7f8e
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-2.A-2.d
@@ -0,0 +1,41 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 2.A-2
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/A-2.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .*
+    Compilation unit name: .*/A-2.c
+#...
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    a ->  80000001: struct A \(size 0x[0-9a-f]*\)
+
+  Types:
+     8[0-9a-f]*: struct A \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct A \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(aligned at 0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 1\) int wombat:32 \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+
+  Strings:
+    0: 
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-2.parent.d b/ld/testsuite/ld-ctf/conflicting-cycle-2.parent.d
new file mode 100644
index 00000000000..8fd84886fd3
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-2.parent.d
@@ -0,0 +1,40 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 2.parent
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0x9b \(0x94 bytes\)
+    String section:	0x9c -- 0xac \(0x11 bytes\)
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    cycle_1 ->  [0-9a-f]*: struct cycle_1 \* \(size 0x[0-9a-f]*\) -> [0-9a-f]*: struct cycle_1 \(size 0x[0-9a-f]*\)
+
+  Types:
+#...
+     [0-9a-f]*: struct cycle_1 \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct cycle_1 \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(aligned at 0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(aligned at 0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct cycle_1 \* next \(aligned at 0x[0-9a-f]*\)
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d b/ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d
new file mode 100644
index 00000000000..fc426de4dcb
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-3.C-1.d
@@ -0,0 +1,39 @@
+#as:
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 3.C-1
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/C.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .*
+    Compilation unit name: .*/C.c
+#...
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    c ->  80000001: struct C \(size 0x[0-9a-f]*\)
+
+  Types:
+     8[0-9a-f]*: struct C \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct C \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(aligned at 0x[0-9a-f]*\)
+
+  Strings:
+    0: 
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d b/ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d
new file mode 100644
index 00000000000..d5e59e30c4c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-3.C-2.d
@@ -0,0 +1,40 @@
+#as:
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Conflicting cycle 3.C-2
+
+.*: +file format .*
+
+#...
+CTF archive member: .*/C-2.c:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Parent name: .*
+    Compilation unit name: .*/C-2.c
+#...
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    c ->  80000001: struct C \(size 0x[0-9a-f]*\)
+
+  Types:
+     8[0-9a-f]*: struct C \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 6\) struct C \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(aligned at 0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 1\) int wombat:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+
+  Strings:
+    0: 
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-cycle-3.parent.d b/ld/testsuite/ld-ctf/conflicting-cycle-3.parent.d
new file mode 100644
index 00000000000..1660821e078
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-cycle-3.parent.d
@@ -0,0 +1,37 @@
+#as:
+#source: A.c
+#source: A-2.c
+#source: B.c
+#source: B-2.c
+#source: C.c
+#source: C-2.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: Conflicting cycle 3
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Type section:	0x0 -- 0x57 \(0x58 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+
+  Types:
+#...
+     [0-9a-f]*: int \[0x0:0x[0-9a-f]*\] \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+#...
+  Strings:
+    0: 
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-enums.d b/ld/testsuite/ld-ctf/conflicting-enums.d
new file mode 100644
index 00000000000..8b16b4cb9e1
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-enums.d
@@ -0,0 +1,35 @@
+#as:
+#source: enum.c
+#source: enum-2.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: Conflicting Enums
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+#...
+  Types:
+
+  Strings:
+#...
+CTF archive member: .*enum.*\.c:
+#...
+  Types:
+     8[0-9a-f]*: enum day_of_the_week \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 8\) enum day_of_the_week \(aligned at 0x[0-9a-f]*\)
+
+  Strings:
+#...
+CTF archive member: .*enum.*\.c:
+#...
+  Types:
+     8[0-9a-f]*: enum day_of_the_week \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x8[0-9a-f]*\) \(kind 8\) enum day_of_the_week \(aligned at 0x[0-9a-f]*\)
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/conflicting-typedefs.d b/ld/testsuite/ld-ctf/conflicting-typedefs.d
new file mode 100644
index 00000000000..09d6c3cf54e
--- /dev/null
+++ b/ld/testsuite/ld-ctf/conflicting-typedefs.d
@@ -0,0 +1,33 @@
+#as:
+#source: typedef-int.c
+#source: typedef-long.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: Conflicting Typedefs
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+#...
+  Types:
+     1: .*int .*
+        .*
+     [0-9]:.*int .*
+        .*
+     [0-9]: word .*
+        \[0x0\] \(ID 0x[0-9]\) \(kind 10\) word \(aligned at 0x[48]\)
+
+  Strings:
+#...
+CTF archive member: .*typedef.*\.c:
+#...
+  Types:
+     80000001: word .*
+        \[0x0\] \(ID 0x80000001\) \(kind 10\) word \(aligned at 0x[48]\)
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/cross-tu-1.c b/ld/testsuite/ld-ctf/cross-tu-1.c
new file mode 100644
index 00000000000..907d071d64e
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-1.c
@@ -0,0 +1,12 @@
+struct B
+{
+  int foo;
+};
+
+struct A
+{
+  long a;
+  struct B *foo;
+};
+
+static struct A *foo __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cross-tu-2.c b/ld/testsuite/ld-ctf/cross-tu-2.c
new file mode 100644
index 00000000000..033d96d3352
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-2.c
@@ -0,0 +1,8 @@
+struct B;
+struct A
+{
+  long a;
+  struct B *foo;
+};
+
+static struct A *foo __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cross-tu-conflicting-2.c b/ld/testsuite/ld-ctf/cross-tu-conflicting-2.c
new file mode 100644
index 00000000000..cfe65a5682a
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-conflicting-2.c
@@ -0,0 +1,8 @@
+struct B;
+struct A
+{
+  int a;
+  struct B *foo;
+};
+
+static struct A *foo __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-1.c b/ld/testsuite/ld-ctf/cross-tu-cyclic-1.c
new file mode 100644
index 00000000000..658e2c6f58d
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-1.c
@@ -0,0 +1,14 @@
+struct A;
+struct B
+{
+  int foo;
+  struct A *bar;
+};
+
+struct A
+{
+  long a;
+  struct B *foo;
+};
+
+static struct A *foo __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-2.c b/ld/testsuite/ld-ctf/cross-tu-cyclic-2.c
new file mode 100644
index 00000000000..aa2d177cf9c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-2.c
@@ -0,0 +1,16 @@
+struct B;
+struct A
+{
+  long a;
+  struct B *foo;
+  struct C *bar;
+};
+
+struct C
+{
+  struct B *foo;
+  int b;
+};
+
+static struct C *foo __attribute__((used));
+static struct A *bar __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-3.c b/ld/testsuite/ld-ctf/cross-tu-cyclic-3.c
new file mode 100644
index 00000000000..19947e85104
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-3.c
@@ -0,0 +1,3 @@
+struct A { struct B *foo; };
+static struct A *a __attribute__((__used__));
+static struct A *conflicty __attribute__((__used__));
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-4.c b/ld/testsuite/ld-ctf/cross-tu-cyclic-4.c
new file mode 100644
index 00000000000..6e0c957e8fe
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-4.c
@@ -0,0 +1,4 @@
+struct A { struct B *foo; };
+struct B { struct B *next; };
+static struct A *a __attribute__((__used__));
+static struct B *conflicty __attribute__((__used__));
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-conflicting.d b/ld/testsuite/ld-ctf/cross-tu-cyclic-conflicting.d
new file mode 100644
index 00000000000..aa36533ea37
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-conflicting.d
@@ -0,0 +1,57 @@
+# Check that types with the same names in distinct TUs show up as
+# conflicting.
+#as:
+#source: cross-tu-cyclic-1.c
+#source: cross-tu-cyclic-2.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: cross-TU-cyclic-conflicting
+
+.*:     file format .*
+
+Contents of CTF section \.ctf:
+
+#...
+  Types:
+#...
+     [0-9a-f]*: long int \[0x0:0x[0-9a-f]*\] \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+#...
+     [0-9a-f]*: struct B .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B .*
+#...
+     [0-9a-f]*: int \[0x0:0x[0-9a-f]*\] \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+#...
+     [0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 9\) struct A .*
+#...
+     [0-9a-f]*: struct C .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct C .*
+#...
+
+  Strings:
+#...
+
+CTF archive member: .*/ld/testsuite/ld-ctf/cross-tu-cyclic-1\.c:
+#...
+  Types:
+     80.*[0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x80.*\) \(kind 6\) struct A .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int a:.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* foo .*
+
+  Strings:
+#...
+
+CTF archive member: .*/ld/testsuite/ld-ctf/cross-tu-cyclic-2\.c:
+#...
+  Types:
+     80.*[0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x80.*\) \(kind 6\) struct A .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int a:.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* foo .*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct C \* bar .*
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d b/ld/testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d
new file mode 100644
index 00000000000..39f5c187e54
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-cyclic-nonconflicting.d
@@ -0,0 +1,50 @@
+# Check that cross-TU chasing to a type in a cycle works.
+# In particular, there should be only one copy of 'struct A *'.
+# (Not entirely reliable: only fails if the two are emitted adjacently.
+# Needs a proper objdump-CTF parser.)
+#as:
+#source: cross-tu-2.c
+#source: cross-tu-cyclic-1.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: cross-TU-cyclic-nonconflicting
+
+.*:     file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+#...
+
+  Labels:
+
+  Data objects:
+#...
+  Function objects:
+
+  Variables:
+#...
+
+  Types:
+#...
+     [0-9a-f]*: struct A \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]\) \(kind 6\) struct A \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int a:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* foo \(aligned at 0x[0-9a-f]*\)
+     [0-9a-f]*: long int .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int:[0-9].*
+     [0-9a-f]*: struct B \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B \(aligned at 0x[0-9a-f]*\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int foo:[0-9]* \(aligned at 0x[0-9a-f]*, format 0x1, offset:bits 0x0:0x[0-9a-f]*\)
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* bar \(aligned at 0x[0-9a-f]*\)
+     [0-9a-f]*: struct B \* \(size 0x[0-9a-f]*\) -> [0-9a-f]*: struct B \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* \(aligned at 0x[0-9a-f]*\)
+     [0-9a-f]*: struct A \* \(size 0x[0-9a-f]*\) -> [0-9a-f]*: struct A \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* \(aligned at 0x[0-9a-f]*\)
+     [0-9a-f]*: int .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int:.*
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/cross-tu-into-cycle.d b/ld/testsuite/ld-ctf/cross-tu-into-cycle.d
new file mode 100644
index 00000000000..6bfdc40a800
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-into-cycle.d
@@ -0,0 +1,64 @@
+# Check that a type outside a cycle pointing into a cycle
+# in another TU does not cause the whole cycle to show up
+# as conflicted.  (Here, we do that by forcing conflicts
+# in the variable section alone, so that we can assert that
+# the type section of any conflicted dicts is empty.)
+# Minimized from libbfd itself.
+#as:
+#source: cross-tu-cyclic-3.c
+#source: cross-tu-cyclic-4.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: cross-TU-into-cycle
+
+.*:     file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+#...
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    a ->  .*
+    conflicty ->  .*
+
+  Types:
+     [0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* foo .*
+     [0-9a-f]*: struct B .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* next .*
+     [0-9a-f]*: struct B \* .*
+        \[0x0\] .*
+     [0-9a-f]*: struct A \* .*
+        \[0x0\] .*
+
+  Strings:
+#...
+
+CTF archive member: .*/ld/testsuite/ld-ctf/cross-tu-cyclic-[34].c:
+
+  Header:
+#...
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+    conflicty ->  .*
+
+  Types:
+
+  Strings:
+#...
diff --git a/ld/testsuite/ld-ctf/cross-tu-noncyclic.d b/ld/testsuite/ld-ctf/cross-tu-noncyclic.d
new file mode 100644
index 00000000000..b90504abb59
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cross-tu-noncyclic.d
@@ -0,0 +1,46 @@
+# Check that noncyclic cross-TU matching-up works.
+# We can guarantee the order of emitted structures, too.
+#as:
+#source: cross-tu-1.c
+#source: cross-tu-2.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: cross-TU-noncyclic
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0x7b \(0x74 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+#...
+  Function objects:
+
+  Variables:
+#...
+
+  Types:
+#...
+     [0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) long int a:[0-9]* .*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* foo .*
+#...
+     [0-9a-f]*: struct B .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B .*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int foo:[0-9]* .*
+#...
+     [0-9a-f]*: struct B \* \(size 0x[0-9a-f]*\) -\> [0-9a-f]*: struct B .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* .*
+#...
+     [0-9a-f]*: struct A \* \(size 0x[0-9a-f]*\) -> [0-9a-f]*: struct A .*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* .*
+#...
diff --git a/ld/testsuite/ld-ctf/ctf.exp b/ld/testsuite/ld-ctf/ctf.exp
new file mode 100644
index 00000000000..9a248f90ec4
--- /dev/null
+++ b/ld/testsuite/ld-ctf/ctf.exp
@@ -0,0 +1,26 @@
+# Copyright (C) 2020 Free Software Foundation, Inc.
+#
+# This file is part of the GNU Binutils.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street - Fifth Floor, Boston,
+# MA 02110-1301, USA.
+#
+
+set ctf_test_list [lsort [glob -nocomplain $srcdir/$subdir/*.d]]
+
+foreach ctf_test $ctf_test_list {
+    verbose [file rootname $ctf_test]
+    run_dump_test [file rootname $ctf_test] { { cc "-gt -fPIC" } }
+}
diff --git a/ld/testsuite/ld-ctf/cycle-1.c b/ld/testsuite/ld-ctf/cycle-1.c
new file mode 100644
index 00000000000..371d411a0d7
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cycle-1.c
@@ -0,0 +1,7 @@
+struct cycle_1 {
+  struct A *a;
+  struct B *b;
+  struct cycle_1 *next;
+};
+
+static struct cycle_1 *cycle_1 __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/cycle-1.d b/ld/testsuite/ld-ctf/cycle-1.d
new file mode 100644
index 00000000000..145221fa99e
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cycle-1.d
@@ -0,0 +1,36 @@
+#as:
+#source: cycle-1.c
+#source: A.c
+#source: B.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Cycle 1
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x1f \(0x20 bytes\)
+    Type section:	0x20 -- 0xc7 \(0xa8 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+  Types:
+#...
+     [0-9a-f]*: struct cycle_1 \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct cycle_1 \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct cycle_1 \* next \(.*
+#...
diff --git a/ld/testsuite/ld-ctf/cycle-2.A.d b/ld/testsuite/ld-ctf/cycle-2.A.d
new file mode 100644
index 00000000000..d224f41683d
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cycle-2.A.d
@@ -0,0 +1,40 @@
+#as:
+#source: A.c
+#source: B.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Cycle 2.A
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x17 \(0x18 bytes\)
+    Type section:	0x18 -- 0x83 \(0x6c bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    a ->  [0-9a-f]*: struct A \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct A \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: A
+#...
diff --git a/ld/testsuite/ld-ctf/cycle-2.B.d b/ld/testsuite/ld-ctf/cycle-2.B.d
new file mode 100644
index 00000000000..598b8ca37d2
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cycle-2.B.d
@@ -0,0 +1,40 @@
+#as:
+#source: A.c
+#source: B.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Cycle 2.B
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x17 \(0x18 bytes\)
+    Type section:	0x18 -- 0x83 \(0x6c bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    b ->  [0-9a-f]*: struct B \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct B \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct C \* c \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: B
+#...
diff --git a/ld/testsuite/ld-ctf/cycle-2.C.d b/ld/testsuite/ld-ctf/cycle-2.C.d
new file mode 100644
index 00000000000..44cdeb56b94
--- /dev/null
+++ b/ld/testsuite/ld-ctf/cycle-2.C.d
@@ -0,0 +1,40 @@
+#as:
+#source: A.c
+#source: B.c
+#source: C.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Cycle 2.C
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x17 \(0x18 bytes\)
+    Type section:	0x18 -- 0x83 \(0x6c bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    c ->  [0-9a-f]*: struct C \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct C \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct C \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: C
+#...
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-0.d b/ld/testsuite/ld-ctf/diag-ctf-version-0.d
new file mode 100644
index 00000000000..4e7402ccd71
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-0.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-ctf-version-0.s
+#ld: -shared
+#name: Diagnostics - CTF version 0
+#warning: CTF section .* not loaded; its types will be discarded: .*
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-0.s b/ld/testsuite/ld-ctf/diag-ctf-version-0.s
new file mode 100644
index 00000000000..5b1c33de1e3
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-0.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x0
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d b/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d
new file mode 100644
index 00000000000..e494810e8ca
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-ctf-version-2-unsupported-feature.s
+#ld: -shared
+#name: Diagnostics - CTF version 2 with unsupported feature
+#warning: CTF section .* not loaded; its types will be discarded: .*
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s b/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s
new file mode 100644
index 00000000000..95e8d5187c1
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-2-unsupported-feature.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x2
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-f.d b/ld/testsuite/ld-ctf/diag-ctf-version-f.d
new file mode 100644
index 00000000000..860aae92a32
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-f.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-ctf-version-f.s
+#ld: -shared
+#name: Diagnostics - Unsupported CTF version
+#warning: CTF section .* not loaded; its types will be discarded: .CTF dict version is too new for libctf.
diff --git a/ld/testsuite/ld-ctf/diag-ctf-version-f.s b/ld/testsuite/ld-ctf/diag-ctf-version-f.s
new file mode 100644
index 00000000000..63b94ca9f6a
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-ctf-version-f.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0xf
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-cttname-invalid.d b/ld/testsuite/ld-ctf/diag-cttname-invalid.d
new file mode 100644
index 00000000000..de4aedc8842
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cttname-invalid.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-cttname-invalid.s
+#ld: -shared
+#name: Diagnostics - Invalid type name.
+#warning: CTF section in .*not loaded; its types will be discarded: .String name offset is corrupt.
diff --git a/ld/testsuite/ld-ctf/diag-cttname-invalid.s b/ld/testsuite/ld-ctf/diag-cttname-invalid.s
new file mode 100644
index 00000000000..dbfdd21fe27
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cttname-invalid.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0xff00
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-cttname-null.d b/ld/testsuite/ld-ctf/diag-cttname-null.d
new file mode 100644
index 00000000000..17df7d6a7bf
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cttname-null.d
@@ -0,0 +1,24 @@
+#as:
+#source: diag-cttname-null.s
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Diagnostics - Null type name
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+#...
+  Variables:
+#...
+    a ->  [0-9a-f]*: struct  \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct  \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct  \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+#...
diff --git a/ld/testsuite/ld-ctf/diag-cttname-null.s b/ld/testsuite/ld-ctf/diag-cttname-null.s
new file mode 100644
index 00000000000..ad6ce60f964
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cttname-null.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-cuname.d b/ld/testsuite/ld-ctf/diag-cuname.d
new file mode 100644
index 00000000000..e243008e0c5
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cuname.d
@@ -0,0 +1,39 @@
+#as:
+#source: diag-cuname.s
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Diagnostics - Invalid CU name offset
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Compilation unit name: \(\?\)
+    Variable section:	.*
+    Type section:	.*
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    a ->  [0-9a-f]*: struct A \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct A \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: \(\?\)
+#...
diff --git a/ld/testsuite/ld-ctf/diag-cuname.s b/ld/testsuite/ld-ctf/diag-cuname.s
new file mode 100644
index 00000000000..dcdbd62aa73
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-cuname.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0
+	.long	0
+	.long	0
+	.long	0xffffffff
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-decompression-failure.d b/ld/testsuite/ld-ctf/diag-decompression-failure.d
new file mode 100644
index 00000000000..c4567e10919
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-decompression-failure.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-decompression-failure.s
+#ld: -shared
+#name: Diagnostics - Decompression failure
+#warning: CTF section.* not loaded; its types will be discarded: .Failed to decompress CTF data.
diff --git a/ld/testsuite/ld-ctf/diag-decompression-failure.s b/ld/testsuite/ld-ctf/diag-decompression-failure.s
new file mode 100644
index 00000000000..40a2fcf66ce
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-decompression-failure.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0x1
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-parlabel.d b/ld/testsuite/ld-ctf/diag-parlabel.d
new file mode 100644
index 00000000000..7e328176cfd
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-parlabel.d
@@ -0,0 +1,39 @@
+#as:
+#source: diag-parlabel.s
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Diagnostics - Non-zero parlabel in parent
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Compilation unit name: .*A.c
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0x37 \(0x30 bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    a ->  [0-9a-f]*: struct A \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct A \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: A
+#...
diff --git a/ld/testsuite/ld-ctf/diag-parlabel.s b/ld/testsuite/ld-ctf/diag-parlabel.s
new file mode 100644
index 00000000000..e0ce57ca535
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-parlabel.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0
+	.long	0x2
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-parname.d b/ld/testsuite/ld-ctf/diag-parname.d
new file mode 100644
index 00000000000..d2ce9aac81f
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-parname.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-parname.s
+#ld: -shared --ctf-variables
+#name: Diagnostics - No parent dictionary
+#warning: CTF linking failed; output will have no CTF section: .The parent CTF dictionary is unavailable.
diff --git a/ld/testsuite/ld-ctf/diag-parname.s b/ld/testsuite/ld-ctf/diag-parname.s
new file mode 100644
index 00000000000..da30e4a7c85
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-parname.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0
+	.long	0
+	.long	0xffffffff
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-unsupported-flag.d b/ld/testsuite/ld-ctf/diag-unsupported-flag.d
new file mode 100644
index 00000000000..879781772aa
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-unsupported-flag.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-unsupported-flag.s
+#ld: -shared
+#name: Diagnostics - Unsupported flag
+#warning: CTF section.* not loaded; its types will be discarded: .CTF header contains flags unknown to libctf.
diff --git a/ld/testsuite/ld-ctf/diag-unsupported-flag.s b/ld/testsuite/ld-ctf/diag-unsupported-flag.s
new file mode 100644
index 00000000000..9e773e91df3
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-unsupported-flag.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff2
+	.byte	0x4
+	.byte	0x80
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/diag-wrong-magic-number-mixed.d b/ld/testsuite/ld-ctf/diag-wrong-magic-number-mixed.d
new file mode 100644
index 00000000000..5415c131940
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-wrong-magic-number-mixed.d
@@ -0,0 +1,39 @@
+#as:
+#source: diag-wrong-magic-number.s
+#source: B.c
+#ld: -shared --ctf-variables
+#name: Diagnostics - Wrong magic number mixed with valid CTF sections
+#warning: CTF section in .* not loaded; its types will be discarded: .Buffer does not contain CTF data.
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Variable section:	0x0 -- 0x17 \(0x18 bytes\)
+    Type section:	0x18 -- 0x83 \(0x6c bytes\)
+    String section:	.*
+
+  Labels:
+
+  Data objects:
+
+  Function objects:
+
+  Variables:
+#...
+    b ->  [0-9a-f]*: struct B \(.*
+#...
+  Types:
+#...
+     [0-9a-f]*: struct B \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct C \* c \(.*
+#...
+  Strings:
+    0: 
+#...
+    [0-9a-f]*: B
+#...
diff --git a/ld/testsuite/ld-ctf/diag-wrong-magic-number.d b/ld/testsuite/ld-ctf/diag-wrong-magic-number.d
new file mode 100644
index 00000000000..847bbf87240
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-wrong-magic-number.d
@@ -0,0 +1,5 @@
+#as:
+#source: diag-wrong-magic-number.s
+#ld: -shared
+#name: Diagnostics - Wrong magic number
+#warning: CTF section in .* not loaded; its types will be discarded: .Buffer does not contain CTF data.
diff --git a/ld/testsuite/ld-ctf/diag-wrong-magic-number.s b/ld/testsuite/ld-ctf/diag-wrong-magic-number.s
new file mode 100644
index 00000000000..a80c8644b19
--- /dev/null
+++ b/ld/testsuite/ld-ctf/diag-wrong-magic-number.s
@@ -0,0 +1,44 @@
+	.file	"A.c"
+	.section	.ctf,"",@progbits
+.Lctf0:
+	.2byte	0xdff3
+	.byte	0x4
+	.byte	0
+	.long	0
+	.long	0
+	.long	0x9
+	.long	0
+	.long	0
+	.long	0x4
+	.long	0x4
+	.long	0x8
+	.long	0x8
+	.long	0x10
+	.long	0x40
+	.long	0x42
+	.long	0x1
+	.long	0x7
+	.long	0x7
+	.long	0x1
+	.long	0x1
+	.long	0x1a000001
+	.long	0x8
+	.long	0x5
+	.long	0
+	.long	0x3
+	.long	0x3
+	.long	0x26000000
+	.long	0x6
+	.long	0
+	.long	0xe000000
+	.long	0x2
+	.ascii "\0"
+	.ascii "A\0"
+	.ascii "B\0"
+	.ascii "b\0"
+	.ascii "a\0"
+	.ascii "/usr/src/binutils-gdb/ld/testsuite/ld-ctf/A.c\0"
+	.text
+	.comm	a,8,8
+	.ident	"GCC: (GNU) 8.3.1 20191121 (Red Hat 8.3.1-5.0.1)"
+	.section	.note.GNU-stack,"",@progbits
diff --git a/ld/testsuite/ld-ctf/enum-2.c b/ld/testsuite/ld-ctf/enum-2.c
new file mode 100644
index 00000000000..f7c095394e3
--- /dev/null
+++ b/ld/testsuite/ld-ctf/enum-2.c
@@ -0,0 +1,3 @@
+enum day_of_the_week {Sunday = 0, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday};
+
+static enum day_of_the_week day __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/enum.c b/ld/testsuite/ld-ctf/enum.c
new file mode 100644
index 00000000000..11851a1fc0f
--- /dev/null
+++ b/ld/testsuite/ld-ctf/enum.c
@@ -0,0 +1,3 @@
+enum day_of_the_week {Monday = 0, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday};
+
+static enum day_of_the_week day __attribute__((used));
diff --git a/ld/testsuite/ld-ctf/function.c b/ld/testsuite/ld-ctf/function.c
new file mode 100644
index 00000000000..806742fc9b5
--- /dev/null
+++ b/ld/testsuite/ld-ctf/function.c
@@ -0,0 +1,3 @@
+int foo (char ch, int i, float f, void * p, void (*bar_ptr)(int)) {
+  return 0;
+}
diff --git a/ld/testsuite/ld-ctf/function.d b/ld/testsuite/ld-ctf/function.d
new file mode 100644
index 00000000000..0ed8eece1bb
--- /dev/null
+++ b/ld/testsuite/ld-ctf/function.d
@@ -0,0 +1,23 @@
+#as:
+#source: function.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: Function
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Compilation unit name: .*function.c
+#...
+    Type section:	0x0 -- 0x8f \(0x90 bytes\)
+    String section:	.*
+#...
+  Types:
+#...
+     [0-9a-f]*: int \(\*\) \(char, int, float, void \*, void \(\*\)\(\*\) \(int\)\) \(size 0x0\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 5\) int \(\*\) \(char, int[0-9]*, float, void \*, void \(\*\)\(\*\) \(int\)\) \(aligned at 0x[0-9a-f]*\)
+#...
diff --git a/ld/testsuite/ld-ctf/slice.c b/ld/testsuite/ld-ctf/slice.c
new file mode 100644
index 00000000000..7937bcf796e
--- /dev/null
+++ b/ld/testsuite/ld-ctf/slice.c
@@ -0,0 +1,6 @@
+struct slices {
+  int one : 1;
+  int two : 2;
+  int six : 6;
+  int ten :10;
+} slices; 
diff --git a/ld/testsuite/ld-ctf/slice.d b/ld/testsuite/ld-ctf/slice.d
new file mode 100644
index 00000000000..b594553568c
--- /dev/null
+++ b/ld/testsuite/ld-ctf/slice.d
@@ -0,0 +1,30 @@
+#as:
+#source: slice.c
+#objdump: --ctf=.ctf
+#ld: -shared --ctf-variables
+#name: Slice
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Compilation unit name: .*slice.c
+    Variable section:	0x0 -- 0x7 \(0x8 bytes\)
+    Type section:	0x8 -- 0xa3 \(0x9c bytes\)
+    String section:	.*
+#...
+  Variables:
+    slices ->  [0-9a-f]*: struct slices \(size 0x4\)
+
+  Types:
+#...
+     [0-9a-f]*: struct slices \(size 0x[0-9a-f]*\)
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct slices \(aligned at 0x1\)
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 1\) int  one:1 \(aligned at 0x1, format 0x1, offset:bits 0x0:0x1\)
+            \[0x1\] \(ID 0x[0-9a-f]*\) \(kind 1\) int  two:2 \(aligned at 0x1, format 0x1, offset:bits 0x1:0x2\)
+            \[0x3\] \(ID 0x[0-9a-f]*\) \(kind 1\) int  six:6 \(aligned at 0x1, format 0x1, offset:bits 0x3:0x6\)
+            \[0x9\] \(ID 0x[0-9a-f]*\) \(kind 1\) int  ten:10 \(aligned at 0x2, format 0x1, offset:bits 0x9:0xa\)
+#...
diff --git a/ld/testsuite/ld-ctf/super-sub-cycles.c b/ld/testsuite/ld-ctf/super-sub-cycles.c
new file mode 100644
index 00000000000..e63411b6227
--- /dev/null
+++ b/ld/testsuite/ld-ctf/super-sub-cycles.c
@@ -0,0 +1,10 @@
+struct A;
+struct B;
+struct D { struct B *b; };
+struct Y;
+struct Z { struct Y *y; struct D *d; };
+struct Y { struct Z z; };
+struct X { struct Y y; };
+struct C { struct A *a; struct D d; };
+struct B { struct C c; struct D d; };
+struct A { struct B b; struct X x; } a;
diff --git a/ld/testsuite/ld-ctf/super-sub-cycles.d b/ld/testsuite/ld-ctf/super-sub-cycles.d
new file mode 100644
index 00000000000..8a46f16f496
--- /dev/null
+++ b/ld/testsuite/ld-ctf/super-sub-cycles.d
@@ -0,0 +1,34 @@
+#as:
+#source: super-sub-cycles.c
+#objdump: --ctf=.ctf
+#ld: -shared
+#name: Super- and sub-cycles
+
+.*: +file format .*
+
+Contents of CTF section .ctf:
+
+  Header:
+    Magic number: dff2
+    Version: 4 \(CTF_VERSION_3\)
+    Compilation unit name: .*super-sub-cycles.c
+#...
+    Type section:	.*\(0x108 bytes\)
+#...
+  Types:
+#...
+     [0-9a-f]*: struct A \(.*
+        \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct A \(.*
+            \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct B b \(.*
+                \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct C c \(.*
+                    \[0x0\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct A \* a \(.*
+                    \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct D d \(.*
+                        \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+                \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct D d \(.*
+                    \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct B \* b \(.*
+            \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct X x \(.*
+                \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct Y y \(.*
+                    \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 6\) struct Z z \(.*
+                        \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct Y \* y \(.*
+                        \[0x[0-9a-f]*\] \(ID 0x[0-9a-f]*\) \(kind 3\) struct D \* d \(.*
+#...
diff --git a/ld/testsuite/ld-ctf/typedef-int.c b/ld/testsuite/ld-ctf/typedef-int.c
new file mode 100644
index 00000000000..15889108e40
--- /dev/null
+++ b/ld/testsuite/ld-ctf/typedef-int.c
@@ -0,0 +1,3 @@
+typedef int word;
+
+static word w;
diff --git a/ld/testsuite/ld-ctf/typedef-long.c b/ld/testsuite/ld-ctf/typedef-long.c
new file mode 100644
index 00000000000..f2e070cc5a4
--- /dev/null
+++ b/ld/testsuite/ld-ctf/typedef-long.c
@@ -0,0 +1,3 @@
+typedef long word;
+
+static word w;
diff --git a/ld/testsuite/ld-ctf/union-1.c b/ld/testsuite/ld-ctf/union-1.c
new file mode 100644
index 00000000000..24d2f2efbb5
--- /dev/null
+++ b/ld/testsuite/ld-ctf/union-1.c
@@ -0,0 +1,4 @@
+union int_or_float {
+  struct child_int * a_child_int;
+  struct child_float * a_child_float;
+} an_int_or_float;
-- 
2.27.0.247.g3dff7de930

[PATCH 58/59] ld, testsuite: only run CTF tests when ld and GCC support CTF

[Nick Alcock]

The CTF testsuite runs GCC to generate CTF that it knows matches the
input .c files before doing a run_dump_test over it.  So we need a GCC
capable of doing that, and we need to always avoid running those tests
if libctf was disabled because the linker will never be capable of it.

ld/
	* configure.ac (enable_libctf): Substitute it.
	* Makefile.am (enablings.exp): New.
	(EXTRA_DEJAGNU_SITE_CONFIG): Add it.
	(DISTCLEANFILES): Likewise.
	* Makefile.in: Regenerate.
	* configure: Likewise.
	* testsuite/lib/ld-lib.exp (compile_one_cc): New.
	(check_ctf_available): Likewise.
	(skip_ctf_tests): Likewise.
	* testsuite/ld-ctf/ctf.exp: Call skip_ctf_tests.
---
 ld/Makefile.am              |  7 +++--
 ld/Makefile.in              |  8 ++++--
 ld/configure                |  6 +++--
 ld/configure.ac             |  1 +
 ld/testsuite/ld-ctf/ctf.exp |  5 ++++
 ld/testsuite/lib/ld-lib.exp | 52 +++++++++++++++++++++++++++++++++++++
 6 files changed, 73 insertions(+), 6 deletions(-)

diff --git a/ld/Makefile.am b/ld/Makefile.am
index 02c4fc16395..bf19f98f1f9 100644
--- a/ld/Makefile.am
+++ b/ld/Makefile.am
@@ -978,6 +978,9 @@ check-DEJAGNU: site.exp
 development.exp: $(BFDDIR)/development.sh
 	$(EGREP) "[development|experimental]=" $(BFDDIR)/development.sh  \
 	  | $(AWK) -F= '{ print "set " $$1 " " $$2 }' > $@
+
+enablings.exp:
+	echo "set enable_libctf ${enable_libctf}" >> $@
 #
 #
 # Build a dummy plugin using libtool.
@@ -1024,7 +1027,7 @@ MAINTAINERCLEANFILES = configdoc.texi ld.1
 CONFIG_STATUS_DEPENDENCIES = $(srcdir)/configure.host $(srcdir)/configure.tgt \
 	$(BFDDIR)/development.sh
 
-EXTRA_DEJAGNU_SITE_CONFIG = development.exp
+EXTRA_DEJAGNU_SITE_CONFIG = development.exp enablings.exp
 
 MOSTLYCLEANFILES = $(STAGESTUFF) ld1$(EXEEXT) ld2$(EXEEXT) ld3$(EXEEXT) \
 	ldemul-list.h crtbegin.@OBJEXT@ crtend.@OBJEXT@ ld.log ld.sum
@@ -1071,7 +1074,7 @@ diststuff: info $(EXTRA_DIST)
 # ld.1 to support parallel build.
 info-recursive: ld.1
 
-DISTCLEANFILES = site.exp development.exp site.bak stringify.sed
+DISTCLEANFILES = site.exp development.exp enablings.exp site.bak stringify.sed
 distclean-local:
 	rm -rf ldscripts
 
diff --git a/ld/Makefile.in b/ld/Makefile.in
index 2fe12e14f63..27350415b30 100644
--- a/ld/Makefile.in
+++ b/ld/Makefile.in
@@ -492,6 +492,7 @@ elf_list_options = @elf_list_options@
 elf_plt_unwind_list_options = @elf_plt_unwind_list_options@
 elf_shlib_list_options = @elf_shlib_list_options@
 enable_initfini_array = @enable_initfini_array@
+enable_libctf = @enable_libctf@
 exec_prefix = @exec_prefix@
 host = @host@
 host_alias = @host_alias@
@@ -1019,7 +1020,7 @@ MAINTAINERCLEANFILES = configdoc.texi ld.1 ld.info
 CONFIG_STATUS_DEPENDENCIES = $(srcdir)/configure.host $(srcdir)/configure.tgt \
 	$(BFDDIR)/development.sh
 
-EXTRA_DEJAGNU_SITE_CONFIG = development.exp
+EXTRA_DEJAGNU_SITE_CONFIG = development.exp enablings.exp
 MOSTLYCLEANFILES = $(STAGESTUFF) ld1$(EXEEXT) ld2$(EXEEXT) ld3$(EXEEXT) \
 	ldemul-list.h crtbegin.@OBJEXT@ crtend.@OBJEXT@ ld.log ld.sum
 
@@ -1030,7 +1031,7 @@ CLEANFILES = dep.sed DEP DEPA DEP1 DEP2 spu_ovl.s spu_ovl.@OBJEXT@ spu_icache.s
 EXTRA_DIST = ldgram.c ldgram.h ldlex.c emultempl/spu_ovl.@OBJEXT@_c \
 	     emultempl/spu_icache.@OBJEXT@_c deffilep.c deffilep.h $(man_MANS)
 
-DISTCLEANFILES = site.exp development.exp site.bak stringify.sed
+DISTCLEANFILES = site.exp development.exp enablings.exp site.bak stringify.sed
 all: $(BUILT_SOURCES) config.h
 	$(MAKE) $(AM_MAKEFLAGS) all-recursive
 
@@ -2555,6 +2556,9 @@ development.exp: $(BFDDIR)/development.sh
 	$(EGREP) "[development|experimental]=" $(BFDDIR)/development.sh  \
 	  | $(AWK) -F= '{ print "set " $$1 " " $$2 }' > $@
 
+enablings.exp:
+	echo "set enable_libctf ${enable_libctf}" >> $@
+
 # DOCUMENTATION TARGETS
 # Manual configuration file; not usually attached to normal configuration,
 # because almost all configs use "gen" version of manual.
diff --git a/ld/configure b/ld/configure
index 11c69bec7b9..9ff13c3c72b 100755
--- a/ld/configure
+++ b/ld/configure
@@ -678,6 +678,7 @@ WARN_WRITE_STRINGS
 NO_WERROR
 WARN_CFLAGS_FOR_BUILD
 WARN_CFLAGS
+enable_libctf
 ENABLE_LIBCTF_FALSE
 ENABLE_LIBCTF_TRUE
 installed_linker
@@ -12037,7 +12038,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 12040 "configure"
+#line 12041 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -12143,7 +12144,7 @@ else
   lt_dlunknown=0; lt_dlno_uscore=1; lt_dlneed_uscore=2
   lt_status=$lt_dlunknown
   cat > conftest.$ac_ext <<_LT_EOF
-#line 12146 "configure"
+#line 12147 "configure"
 #include "confdefs.h"
 
 #if HAVE_DLFCN_H
@@ -15958,6 +15959,7 @@ fi
 
 
 
+
 # Set the 'development' global.
 . $srcdir/../bfd/development.sh
 
diff --git a/ld/configure.ac b/ld/configure.ac
index 5a4938afdb0..5ef25e3eb88 100644
--- a/ld/configure.ac
+++ b/ld/configure.ac
@@ -226,6 +226,7 @@ if test "${enable_libctf}" = yes; then
     AC_DEFINE(ENABLE_LIBCTF, 1, [Handle .ctf type-info sections])
 fi
 AM_CONDITIONAL(ENABLE_LIBCTF, test "${enable_libctf}" = yes)
+AC_SUBST(enable_libctf)
 
 AM_BINUTILS_WARNINGS
 
diff --git a/ld/testsuite/ld-ctf/ctf.exp b/ld/testsuite/ld-ctf/ctf.exp
index 9a248f90ec4..3e44de7aaab 100644
--- a/ld/testsuite/ld-ctf/ctf.exp
+++ b/ld/testsuite/ld-ctf/ctf.exp
@@ -18,6 +18,11 @@
 # MA 02110-1301, USA.
 #
 
+if [skip_ctf_tests] {
+    unsupported "no CTF format support in the compiler, or CTF disabled"
+    return 0
+}
+
 set ctf_test_list [lsort [glob -nocomplain $srcdir/$subdir/*.d]]
 
 foreach ctf_test $ctf_test_list {
diff --git a/ld/testsuite/lib/ld-lib.exp b/ld/testsuite/lib/ld-lib.exp
index 8d851a94b71..8d476ca48f3 100644
--- a/ld/testsuite/lib/ld-lib.exp
+++ b/ld/testsuite/lib/ld-lib.exp
@@ -1569,3 +1569,55 @@ proc check_gnu2_tls_available { } {
     }
     return $gnu2_tls_available_saved
 }
+
+# Compile a C source file, with the specified additional_flags.
+proc compile_one_cc { src output additional_flags } {
+    global CC
+    global CFLAGS
+
+    set flags ""
+    if [board_info [target_info name] exists cflags] {
+	append flags " [board_info [target_info name] cflags]"
+    }
+    if [board_info [target_info name] exists ldflags] {
+	append flags " [board_info [target_info name] ldflags]"
+    }
+
+    if [is_remote host] {
+	set src [remote_download host $src]
+    }
+    return [run_host_cmd_yesno "$CC" "$flags $CFLAGS $additional_flags $src -o $output"]
+}
+
+# Returns true if the target compiler supports -gt
+proc check_ctf_available { } {
+    global ctf_available_saved
+
+    if {![info exists ctf_available_saved]} {
+	set basename "tmpdir/ctf_available[pid]"
+	set src ${basename}.c
+	set output ${basename}.o
+	set f [open $src "w"]
+	puts $f "int main() { return 0; }"
+	close $f
+	set ctf_available_saved [compile_one_cc $src $output "-gt -c"]
+	remote_file host delete $src
+	remote_file host delete $output
+	file delete $src
+    }
+    return $ctf_available_saved
+}
+
+proc skip_ctf_tests { } {
+    global enable_libctf
+
+    if {$enable_libctf eq "no"} {
+	return 1
+    }
+
+    if [check_ctf_available] {
+	return 0
+    }
+
+    return 1
+}
-- 
2.27.0.247.g3dff7de930

[PATCH 59/59] ld: do not produce one empty output .ctf section for every input .ctf

[Nick Alcock]

The trick we use to prevent ld doing as it does for almost all other
sections and copying the input CTF section into the output has recently
broken, causing output to be produced with a valid CTF section followed
by massive numbers of CTF sections, one per .ctf in the input (minus
one, for the one that was filled out by ctf_link).  Their size is being
forcibly set to zero, but they're still present, wasting space and
looking ridiculous.

This is not right:

ld/ld-new  :
section                     size      addr
.interp                       28   4194984
[...]
.bss                       21840   6788544
.comment                      92         0
.ctf                       87242         0
.ctf                           0         0
.ctf                           0         0
[snip 131 more empty sections]
.gnu.build.attributes       7704   6818576
.debug_aranges              6592         0
.debug_info              4488859         0
.debug_abbrev             150099         0
.debug_line               796759         0
.debug_str                237926         0
.debug_loc               2247302         0
.debug_ranges             237920         0
Total                   10865285

The fix is to exclude these unwanted input sections from being present
in the output.  We tried this before and it broke things, because if you
exclude all the .ctf sections there isn't going to be one in the output
so there is nowhere to put the deduplicated CTF. The solution to that is
really simple: set SEC_EXCLUDE on *all but one* CTF section.  We don't
care which one (they're all the same once their size has been zeroed),
so just pick the first we see.

ld/
	* ldlang.c (ldlang_open_ctf): Set SEC_EXCLUDE on all but the
	first input .ctf section.
---
 ld/ldlang.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/ld/ldlang.c b/ld/ldlang.c
index da89838ed3d..814880b16c1 100644
--- a/ld/ldlang.c
+++ b/ld/ldlang.c
@@ -3701,12 +3701,15 @@ ldlang_open_ctf (void)
 	}
 
       /* Prevent the contents of this section from being written, while
-	 requiring the section itself to be duplicated in the output.  */
+	 requiring the section itself to be duplicated in the output, but only
+	 once.  */
       /* This section must exist if ctf_bfdopen() succeeded.  */
       sect = bfd_get_section_by_name (file->the_bfd, ".ctf");
       sect->size = 0;
       sect->flags |= SEC_NEVER_LOAD | SEC_HAS_CONTENTS | SEC_LINKER_CREATED;
 
+      if (any_ctf)
+	sect->flags |= SEC_EXCLUDE;
       any_ctf = 1;
     }
 
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 46/59] ctf, link: fix spurious conflicts of variables in the variable section

[Nick Alcock]

On 1 Jul 2020, Nick Alcock via Binutils outgrape:

> When we link a CTF variable, we check to see if it already exists in the
> parent dict first: if it does, and it has a type the same as the type we
> would populate it with, we assume we don't need to do anything:
> otherwise, we populate it in a per-CU child.

Well, OK, this commit log is getting changed. It stuck out a mile in the
mailing list summary view :)

'ctf'? No, libctf!

Re: [PATCH 00/59] Deduplicating CTF linker

[Nick Alcock]

On 1 Jul 2020, Nick Alcock via Binutils said:

> So the deduplicating CTF linker is finally ready for review.  (Note: ready
> for *review*, probably not yet quite ready for committing, since to speed
> things up a bit I've posted before all non-Linux-arch tests are finished.
> In particular, big-endian tests, Solaris tests, and tests on mingw and
> cygwin will be done in parallel with this review.  Cross build-and-checks to
> every supported arch, --enable-targets=all tests, and 64-bit and 32-bit
> tests on x86-64 Linux and FreeBSD have been done already.)
>
> Please read and review as you wish -- most of these are libctf-only changes,
> but I'd be very happy for review of those too.  There are only a few dozen
> lines of non-testsuite changes outside libctf.

Ping? Patches 53, 54, 55, 56, 59 and possibly 57 and 58 are outside
libctf and thus are awaiting review. (And I'd be happy with reviews for
any of the others, too.)

I understand it's near release time and everyone's concentrating on
that, and I'm quite happy to wait until after release: I'm just hoping
this hasn't been entirely forgotten by everyone :)

Thanks!


(There will be at least one more round of this series before pushing:
there are a bunch of small fixes that fell out of cygwin and mingw
testing. This series is long, so a repost of the whole thing with the
fixes is probably not wanted: I'm not sure whether people would prefer
fixups to the existing commits, a range-diff, or a simple repost of the
corrected commits with changes notated in the commit log.)

Re: [PATCH 00/59] Deduplicating CTF linker

[Alan Modra]

On Tue, Jul 14, 2020 at 10:31:27PM +0100, Nick Alcock via Binutils wrote:
> On 1 Jul 2020, Nick Alcock via Binutils said:
> 
> > So the deduplicating CTF linker is finally ready for review.  (Note: ready
> > for *review*, probably not yet quite ready for committing, since to speed
> > things up a bit I've posted before all non-Linux-arch tests are finished.
> > In particular, big-endian tests, Solaris tests, and tests on mingw and
> > cygwin will be done in parallel with this review.  Cross build-and-checks to
> > every supported arch, --enable-targets=all tests, and 64-bit and 32-bit
> > tests on x86-64 Linux and FreeBSD have been done already.)
> >
> > Please read and review as you wish -- most of these are libctf-only changes,
> > but I'd be very happy for review of those too.  There are only a few dozen
> > lines of non-testsuite changes outside libctf.
> 
> Ping? Patches 53, 54, 55, 56, 59 and possibly 57 and 58 are outside

These are OK.

> libctf and thus are awaiting review. (And I'd be happy with reviews for
> any of the others, too.)
> 
> I understand it's near release time and everyone's concentrating on
> that, and I'm quite happy to wait until after release: I'm just hoping
> this hasn't been entirely forgotten by everyone :)
> 
> Thanks!
> 
> 
> (There will be at least one more round of this series before pushing:
> there are a bunch of small fixes that fell out of cygwin and mingw
> testing. This series is long, so a repost of the whole thing with the
> fixes is probably not wanted: I'm not sure whether people would prefer
> fixups to the existing commits, a range-diff, or a simple repost of the
> corrected commits with changes notated in the commit log.)

Deltas from the previously posted patches would be good.  That should
be easy if you keep fixups separate, aiming to squash fixups before
committing.

-- 
Alan Modra
Australia Development Lab, IBM

Re: [PATCH 00/59] Deduplicating CTF linker

[Nick Alcock]

On 20 Jul 2020, Alan Modra via Binutils verbalised:

> On Tue, Jul 14, 2020 at 10:31:27PM +0100, Nick Alcock via Binutils wrote:
>> On 1 Jul 2020, Nick Alcock via Binutils said:
>> 
[...]
>> > Please read and review as you wish -- most of these are libctf-only changes,
>> > but I'd be very happy for review of those too.  There are only a few dozen
>> > lines of non-testsuite changes outside libctf.
>> 
>> Ping? Patches 53, 54, 55, 56, 59 and possibly 57 and 58 are outside
>
> These are OK.

Thank you! This gets libctf a lot closer to being actually usable by gdb
in real programs :) (One more feature is needed, which I'm working on
now.)

>> (There will be at least one more round of this series before pushing:
>> there are a bunch of small fixes that fell out of cygwin and mingw
>> testing. This series is long, so a repost of the whole thing with the
>> fixes is probably not wanted: I'm not sure whether people would prefer
>> fixups to the existing commits, a range-diff, or a simple repost of the
>> corrected commits with changes notated in the commit log.)
>
> Deltas from the previously posted patches would be good.  That should
> be easy if you keep fixups separate, aiming to squash fixups before
> committing.

Will do. All the fixups are in libctf/ only, or in the new CTF
testsuite, so I don't think they'll need much in the way of review.

Re: [PATCH 27/59] fixup! libctf, next: introduce new class of easier-to-use iterators

[Nick Alcock]

Don't post-modify here: GCC complains.
---
 libctf/ctf-archive.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 2e386a02dcf..4fbb246c67b 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -836,7 +836,8 @@ ctf_archive_next (const ctf_archive_t *wrapper, ctf_next_t **it, const char **na
 					 + sizeof (struct ctf_archive));
       nametbl = (((const char *) arc) + le64toh (arc->ctfa_names));
 
-      name_ = &nametbl[le64toh (modent[i->ctn_n++].name_offset)];
+      name_ = &nametbl[le64toh (modent[i->ctn_n].name_offset)];
+      i->ctn_n++;
     } while (skip_parent && strcmp (name_, _CTF_SECTION) == 0);
 
   if (name)
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 49/59] fixup! libctf, dedup: add new configure option --enable-libctf-hash-debugging

[Nick Alcock]

Add missing brackets to this preprocessor conditional.
---
 libctf/ctf-impl.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-impl.h b/libctf/ctf-impl.h
index 06b98da2a9d..35320d46f66 100644
--- a/libctf/ctf-impl.h
+++ b/libctf/ctf-impl.h
@@ -71,7 +71,7 @@ extern "C"
 
 #endif
 
-#if defined ENABLE_LIBCTF_HASH_DEBUGGING && !defined (NDEBUG)
+#if defined (ENABLE_LIBCTF_HASH_DEBUGGING) && !defined (NDEBUG)
 #include <assert.h>
 #define ctf_assert(fp, expr) (assert (expr), 1)
 #else
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 02/59] fixup! libctf: restructure error handling to reduce relocations

[Nick Alcock]

We can't use _S as a macro name since it collides with a name in
Cygwin's <ctype.h>.

Use a less-collidy name instead.
---
 libctf/ctf-error.c  | 12 ++++++------
 libctf/mkerrors.sed |  4 ++--
 2 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-error.c b/libctf/ctf-error.c
index ac42784df65..20971f4275b 100644
--- a/libctf/ctf-error.c
+++ b/libctf/ctf-error.c
@@ -37,17 +37,17 @@ static const union _ctf_errlist_t
 {
   __extension__ struct
   {
-#define _S(n, s) char ERRSTRFIELD (__LINE__) [sizeof (s)];
+#define _CTF_STR(n, s) char ERRSTRFIELD (__LINE__) [sizeof (s)];
 #include "ctf-error.h"
-#undef _S
+#undef _CTF_STR
   };
   char str[1];
 } _ctf_errlist =
   {
    {
-#define _S(n, s) s,
+#define _CTF_STR(n, s) s,
 #include "ctf-error.h"
-#undef _S
+#undef _CTF_STR
    }
   };
 
@@ -55,9 +55,9 @@ static const union _ctf_errlist_t
 
 static const unsigned int _ctf_erridx[] =
   {
-#define _S(n, s) [n - ECTF_BASE] = offsetof (union _ctf_errlist_t, ERRSTRFIELD (__LINE__)),
+#define _CTF_STR(n, s) [n - ECTF_BASE] = offsetof (union _ctf_errlist_t, ERRSTRFIELD (__LINE__)),
 #include "ctf-error.h"
-#undef _S
+#undef _CTF_STR
   };
 
 const char *
diff --git a/libctf/mkerrors.sed b/libctf/mkerrors.sed
index 146fd4abc1d..ddd4d2286e7 100644
--- a/libctf/mkerrors.sed
+++ b/libctf/mkerrors.sed
@@ -22,7 +22,7 @@
     /^ *ECTF_/!n;
     # Strip out the base initializer.
     s, = ECTF_BASE,,;
-    # Transform errors into _S(...).
-    s@^ *\(ECTF_[^[:blank:],]*\),\{0,1\}[[:blank:]]*/\* \(.*\).  \*/$@_S (\1, "\2")@;
+    # Transform errors into _STR(...).
+    s@^ *\(ECTF_[^[:blank:],]*\),\{0,1\}[[:blank:]]*/\* \(.*\).  \*/$@_CTF_STR (\1, "\2")@;
     p;
   }
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 58/59] fixup! ld, testsuite: only run CTF tests when ld and GCC support CTF

[Nick Alcock]

Require ELF support for diags tests, too.  (This is possibly not
restrictive enough.)
---
 ld/testsuite/ld-ctf/ctf.exp | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/ld/testsuite/ld-ctf/ctf.exp b/ld/testsuite/ld-ctf/ctf.exp
index 3e44de7aaab..5d177afef0d 100644
--- a/ld/testsuite/ld-ctf/ctf.exp
+++ b/ld/testsuite/ld-ctf/ctf.exp
@@ -26,6 +26,11 @@ if [skip_ctf_tests] {
 set ctf_test_list [lsort [glob -nocomplain $srcdir/$subdir/*.d]]
 
 foreach ctf_test $ctf_test_list {
+    if [string equal -length [string length "diag-"] "diag-" [file tail $ctf_test]] {
+	if ![is_elf_format] {
+	    continue
+	}
+    }
     verbose [file rootname $ctf_test]
     run_dump_test [file rootname $ctf_test] { { cc "-gt -fPIC" } }
 }
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 50/59] squash! libctf, dedup: add deduplicator

[Nick Alcock]

Build on systems lacking stpcpy, such as mingw.

ChangeLog delta added to original commit:

libctf/
	* configure.ac: Check for stpcpy.
        * config.h.in: Regenerated.
        * configure: Regenerated.
        * ctf-decls.h [!HAVE_DECL_STPCPY]: Add prototype.
---
 libctf/config.h.in  |  4 ++++
 libctf/configure    | 10 ++++++++++
 libctf/configure.ac |  2 +-
 libctf/ctf-decls.h  |  4 ++++
 4 files changed, 19 insertions(+), 1 deletion(-)

diff --git a/libctf/config.h.in b/libctf/config.h.in
index 2ddd5126088..5db637c7903 100644
--- a/libctf/config.h.in
+++ b/libctf/config.h.in
@@ -32,6 +32,10 @@
    don't. */
 #undef HAVE_DECL_BSWAP_64
 
+/* Define to 1 if you have the declaration of `stpcpy', and to 0 if you don't.
+   */
+#undef HAVE_DECL_STPCPY
+
 /* Define to 1 if you have the declaration of `vasprintf', and to 0 if you
    don't. */
 #undef HAVE_DECL_VASPRINTF
diff --git a/libctf/configure b/libctf/configure
index 9f16ae92b8d..a1f5b7ff09f 100755
--- a/libctf/configure
+++ b/libctf/configure
@@ -13186,6 +13186,16 @@ fi
 cat >>confdefs.h <<_ACEOF
 #define HAVE_DECL_VASPRINTF $ac_have_decl
 _ACEOF
+ac_fn_c_check_decl "$LINENO" "stpcpy" "ac_cv_have_decl_stpcpy" "$ac_includes_default"
+if test "x$ac_cv_have_decl_stpcpy" = xyes; then :
+  ac_have_decl=1
+else
+  ac_have_decl=0
+fi
+
+cat >>confdefs.h <<_ACEOF
+#define HAVE_DECL_STPCPY $ac_have_decl
+_ACEOF
 
 
 
diff --git a/libctf/configure.ac b/libctf/configure.ac
index 3799a0cb906..f5ed973b10d 100644
--- a/libctf/configure.ac
+++ b/libctf/configure.ac
@@ -108,7 +108,7 @@ AC_CHECK_FUNCS(pread)
 
 dnl Check for bswap_{16,32,64}
 AC_CHECK_DECLS([bswap_16, bswap_32, bswap_64], [], [], [[#include <byteswap.h>]])
-AC_CHECK_DECLS([asprintf, vasprintf])
+AC_CHECK_DECLS([asprintf, vasprintf, stpcpy])
 
 dnl Check for qsort_r.  (Taken from gnulib.)
 AC_CHECK_FUNCS_ONCE([qsort_r])
diff --git a/libctf/ctf-decls.h b/libctf/ctf-decls.h
index c47a72e722f..e1ada3b9592 100644
--- a/libctf/ctf-decls.h
+++ b/libctf/ctf-decls.h
@@ -72,4 +72,8 @@ void ctf_qsort_r (void *base, size_t nmemb, size_t size,
 #define MAX(a, b) ((a) > (b) ? (a) : (b))
 #define MIN(a, b) ((a) < (b) ? (a) : (b))
 
+#if !HAVE_DECL_STPCPY
+extern char *stpcpy (char *, const char *);
+#endif
+
 #endif /* _CTF_DECLS_H */
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 50/59] fixup! libctf, dedup: add deduplicator

[Nick Alcock]

Fix a confusing comment (ctf_dedup_hash_type was once split in two, and
this comment is a relic of that era).
---
 libctf/ctf-dedup.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/libctf/ctf-dedup.c b/libctf/ctf-dedup.c
index c7badcd17a1..670d14f77fb 100644
--- a/libctf/ctf-dedup.c
+++ b/libctf/ctf-dedup.c
@@ -971,10 +971,7 @@ ctf_dedup_rhash_type (ctf_file_t *fp, ctf_file_t *input, ctf_file_t **inputs,
   return NULL;
 }
 
-/* Internal implementation of ctf_dedup_hash_type (below), which is the entry
-   point others should call.
-
-   Hash a TYPE in the INPUT: FP is the eventual output, where the ctf_dedup
+/* Hash a TYPE in the INPUT: FP is the eventual output, where the ctf_dedup
    state is stored.  INPUT_NUM is the number of this input in the set of inputs.
    Record its hash in FP's cd_type_hashes once it is known.  PARENTS is
    described in the comment above ctf_dedup.
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 50/59] fixup! libctf, dedup: add deduplicator

[Nick Alcock]

Fixes for systems on which sizeof (void *) > sizeof (long).

This is all quite ugly: perhaps some new wrappers aroud
ctf_dynhash_lookup and ctf_dynhash_*insert() are called for.
---
 libctf/ctf-dedup.c | 19 ++++++++++---------
 libctf/ctf-hash.c  |  2 +-
 2 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/libctf/ctf-dedup.c b/libctf/ctf-dedup.c
index 670d14f77fb..77b34f4539d 100644
--- a/libctf/ctf-dedup.c
+++ b/libctf/ctf-dedup.c
@@ -1269,10 +1269,10 @@ ctf_dedup_populate_mappings (ctf_file_t *fp, ctf_file_t *input _libctf_unused_,
     }
 
   /* This will, conveniently, return NULL (i.e. 0) for a new entry.  */
-  count = (long int) ctf_dynhash_lookup (name_counts, hval);
+  count = (long int) (uintptr_t) ctf_dynhash_lookup (name_counts, hval);
 
   if (ctf_dynhash_cinsert (name_counts, hval,
-			  (const void *) (count + 1)) < 0)
+			   (const void *) (uintptr_t) (count + 1)) < 0)
     return ctf_set_errno (fp, errno);
 
   return 0;
@@ -1503,9 +1503,9 @@ ctf_dedup_detect_name_ambiguity (ctf_file_t *fp, ctf_file_t **inputs)
 	  while ((err = ctf_dynhash_cnext (name_counts, &j, &key, &count)) == 0)
 	    {
 	      hval = (const char *) key;
-	      if ((long int) count > max_hcount)
+	      if ((long int) (uintptr_t) count > max_hcount)
 		{
-		  max_hcount = (long int) count;
+		  max_hcount = (long int) (uintptr_t) count;
 		  max_hval = hval;
 		}
 	    }
@@ -1866,7 +1866,7 @@ ctf_dedup (ctf_file_t *output, ctf_file_t **inputs, uint32_t ninputs,
   ctf_next_t *it = NULL;
 
   for (i = 0; i < ninputs; i++)
-    ctf_dprintf ("Input %zi: %s\n", i, ctf_link_input_name (inputs[i]));
+    ctf_dprintf ("Input %i: %s\n", (int) i, ctf_link_input_name (inputs[i]));
 
   if (ctf_dedup_init (output) < 0)
     return -1; 					/* errno is set for us.  */
@@ -2396,7 +2396,8 @@ ctf_dedup_maybe_synthesize_forward (ctf_file_t *output, ctf_file_t *target,
 	}
 
       if (ctf_dynhash_cinsert (td->cd_output_emission_conflicted_forwards,
-			       decorated, (void *) emitted_forward) < 0)
+			       decorated, (void *) (uintptr_t)
+			       emitted_forward) < 0)
 	{
 	  ctf_set_errno (output, ENOMEM);
 	  return CTF_ERR;
@@ -2527,7 +2528,7 @@ ctf_dedup_id_to_target (ctf_file_t *output, ctf_file_t *target,
     }
   if (!ctf_assert (output, target_id))
     return -1;
-  return (ctf_id_t) target_id;
+  return (ctf_id_t) (uintptr_t) target_id;
 }
 
 /* Emit a single deduplicated TYPE with the given HVAL, located in a given
@@ -2857,7 +2858,7 @@ ctf_dedup_emit_type (const char *hval, ctf_file_t *output, ctf_file_t **inputs,
   if (!emission_hashed
       && new_type != 0
       && ctf_dynhash_cinsert (target->ctf_dedup.cd_output_emission_hashes,
-			      hval, (void *) new_type) < 0)
+			      hval, (void *) (uintptr_t) new_type) < 0)
     {
       ctf_err_warn (output, 0, "out of memory tracking deduplicated "
 		    "global type IDs");
@@ -3016,7 +3017,7 @@ ctf_dedup_populate_type_mapping (ctf_file_t *shared, ctf_file_t *fp,
 				  &i, &k, &v)) == 0)
     {
       const char *hval = (const char *) k;
-      ctf_id_t id_out = (ctf_id_t) v;
+      ctf_id_t id_out = (ctf_id_t) (uintptr_t) v;
       ctf_next_t *j = NULL;
       ctf_dynset_t *type_ids;
       const void *id;
diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 76572ff96e4..fd5d61ed7b8 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -124,7 +124,7 @@ ctf_hash_type_id_key (const void *ptr)
   ctf_type_id_key_t *k = (ctf_type_id_key_t *) hep->key;
 
   return htab_hash_pointer ((void *) (uintptr_t) k->ctii_input_num)
-    + 59 * htab_hash_pointer ((void *) k->ctii_type);
+    + 59 * htab_hash_pointer ((void *) (uintptr_t) k->ctii_type);
 }
 
 int
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 40/59] fixup! libctf: rename the type_mapping_key to type_key

[Nick Alcock]

Fixes for systems on which sizeof (void *) > sizeof (long).
---
 libctf/ctf-hash.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index fd5d61ed7b8..297526094cc 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -101,7 +101,8 @@ ctf_hash_type_key (const void *ptr)
   ctf_helem_t *hep = (ctf_helem_t *) ptr;
   ctf_link_type_key_t *k = (ctf_link_type_key_t *) hep->key;
 
-  return htab_hash_pointer (k->cltk_fp) + 59 * htab_hash_pointer ((void *) k->cltk_idx);
+  return htab_hash_pointer (k->cltk_fp) + 59
+    * htab_hash_pointer ((void *) (uintptr_t) k->cltk_idx);
 }
 
 int
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 52/59] fixup! libctf, link: tie in the deduplicating linker

[Nick Alcock]

We can't use -1 here without causing trouble on platforms with 32-bit
ssize_t.  We also have to cast the input to labs to long int to allow
for platforms on which ssize_t is long long (it's easier to do that
than it is to use llabs, which is less common: counts of inputs that
exceed 32 bits are prohibited anyway so we don't care about any
precision loss that high up.)

So much work for such an unlikely error case :)
---
 libctf/ctf-link.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libctf/ctf-link.c b/libctf/ctf-link.c
index fd7298749cd..f269fe7ac0e 100644
--- a/libctf/ctf-link.c
+++ b/libctf/ctf-link.c
@@ -1176,14 +1176,15 @@ ctf_link_deduplicating_per_cu (ctf_file_t *fp)
       uint32_t noutputs;
       uint32_t *parents;
 
-      if ((ninputs = ctf_link_deduplicating_count_inputs (fp, in, &only_input)) < 0)
+      if ((ninputs = ctf_link_deduplicating_count_inputs (fp, in,
+							  &only_input)) == -1)
 	goto err_open_inputs;
 
       /* CU mapping with no inputs?  Skip.  */
       if (ninputs == 0)
 	continue;
 
-      if (ninputs > -1)
+      if (labs ((long int) ninputs) > 0xfffffffe)
 	{
 	  ctf_err_warn (fp, 0, "Too many inputs in deduplicating link: %li",
 			(long int) ninputs);
-- 
2.27.0.247.g3dff7de930

[PATCH 0/4] fallout of various portability testing

[Nick Alcock]

These patches are non-fixups that fell out of tests on bigendian Linux,
64-bit Cygwin, and 32- and 64-bit MinGW tests (accompanying the fixups I
just posted, but not applying to patches in the deduplicator series, or
things I think should be separate commits). A cross of x86_64-pc-cygwin64 ->
x86_64-pc-linux-gnu is building at the moment so I can try to actually test
the CTF deduplicator on 64-bit Windows, even if only in cross.  That's
because unfortunately it seems that more linker work is needed before
non-ELF platforms (like Cygwin and mingw) will actually emit CTF sections:
right now the deduplicator runs and the linker never emits the section
afterwards.

I'll get to that, but for now just (temporarily) suppress CTF tests, even
with a suitable compiler, unless this is an ELF platform, on the grounds
that if PE is failing to emit the .ctf section, everything else I haven't
paid attention to yet is likely to as well.

Nick Alcock (4):
  ld, testsuite: do not run CTF tests at all on non-ELF for now
  libctf, binutils: fix big-endian libctf archive opening
  libctf: fix isspace casts
  libctf: fixes for systems on which sizeof (void *) > sizeof (long)

 ld/testsuite/ld-ctf/ctf.exp |  5 +++++
 libctf/ctf-archive.c        |  2 +-
 libctf/ctf-create.c         | 16 ++++++++++------
 libctf/ctf-hash.c           |  2 +-
 libctf/ctf-lookup.c         |  6 +++---
 libctf/ctf-types.c          |  2 +-
 6 files changed, 21 insertions(+), 12 deletions(-)

-- 
2.27.0.247.g3dff7de930

[PATCH 1/4] ld, testsuite: do not run CTF tests at all on non-ELF for now

[Nick Alcock]

Right now, the linker is not emitting CTF sections on (at least some)
non-ELF platforms, because work similar to that done for ELF needs to be
done to each platform in turn to emit linker-generated sections whose
contents are programmatically derived.  (Or something better needs to be
done.)

So, for now, the CTF tests will fail on non-ELF for lack of a .ctf
section in the output: so skip the CTF tests there temporarily.
(This is not the same as the permanent skip of the diags tests, which is
done because the input for those is assembler that depends on the ELF
syntax of pseudos like .section: this is only a temporary skip, until
the linker grows support for CTF on more targets.)

ld/
	* testsuite/ld-ctf/ctf.exp: Skip on non-ELF for now.
---
 ld/testsuite/ld-ctf/ctf.exp | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/ld/testsuite/ld-ctf/ctf.exp b/ld/testsuite/ld-ctf/ctf.exp
index 5d177afef0d..be4c6ed3d6e 100644
--- a/ld/testsuite/ld-ctf/ctf.exp
+++ b/ld/testsuite/ld-ctf/ctf.exp
@@ -23,6 +23,11 @@ if [skip_ctf_tests] {
     return 0
 }
 
+if ![is_elf_format] {
+    unsupported "CTF needs bfd changes to be emitted on non-ELF"
+    return 0
+}
+
 set ctf_test_list [lsort [glob -nocomplain $srcdir/$subdir/*.d]]
 
 foreach ctf_test $ctf_test_list {
-- 
2.27.0.247.g3dff7de930

[PATCH 2/4] libctf, binutils: fix big-endian libctf archive opening

[Nick Alcock]

The recent commit "libctf, binutils: support CTF archives like objdump"
broke opening of CTF archives on big-endian platforms.

This didn't affect anyone much before now because the linker never
emitted CTF archives because it wasn't detecting ambiguous types
properly: now it does, and this bug becomes obvious.

Fix trivial.

libctf/
	* ctf-archive.c (ctf_arc_bufopen): Endian-swap the archive magic
	number if needed.
---
 libctf/ctf-archive.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libctf/ctf-archive.c b/libctf/ctf-archive.c
index 3b4304f0894..4fbb246c67b 100644
--- a/libctf/ctf-archive.c
+++ b/libctf/ctf-archive.c
@@ -393,7 +393,7 @@ ctf_arc_bufopen (const ctf_sect_t *ctfsect, const ctf_sect_t *symsect,
   ctf_file_t *fp = NULL;
 
   if (ctfsect->cts_size > sizeof (uint64_t) &&
-      ((*(uint64_t *) ctfsect->cts_data) == CTFA_MAGIC))
+      (le64toh ((*(uint64_t *) ctfsect->cts_data)) == CTFA_MAGIC))
     {
       /* The archive is mmappable, so this operation is trivial.
 
-- 
2.27.0.247.g3dff7de930

[PATCH 3/4] libctf: fix isspace casts

[Nick Alcock]

isspace() notoriously takes an int, not a char.  Cast uses
appropriately.

libctf/
	* ctf-lookup.c (ctf_lookup_by_name): Adjust.
---
 libctf/ctf-lookup.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/libctf/ctf-lookup.c b/libctf/ctf-lookup.c
index 8daab632dca..57fb6c580ee 100644
--- a/libctf/ctf-lookup.c
+++ b/libctf/ctf-lookup.c
@@ -83,7 +83,7 @@ ctf_lookup_by_name (ctf_file_t *fp, const char *name)
 
   for (p = name, end = name + strlen (name); *p != '\0'; p = q)
     {
-      while (isspace (*p))
+      while (isspace ((int) *p))
 	p++;			/* Skip leading whitespace.  */
 
       if (p == end)
@@ -133,13 +133,13 @@ ctf_lookup_by_name (ctf_file_t *fp, const char *name)
 	       strncmp (p, lp->ctl_prefix, (size_t) (q - p)) == 0) &&
 	      (size_t) (q - p) >= lp->ctl_len)
 	    {
-	      for (p += lp->ctl_len; isspace (*p); p++)
+	      for (p += lp->ctl_len; isspace ((int) *p); p++)
 		continue;	/* Skip prefix and next whitespace.  */
 
 	      if ((q = strchr (p, '*')) == NULL)
 		q = end;	/* Compare until end.  */
 
-	      while (isspace (q[-1]))
+	      while (isspace ((int) q[-1]))
 		q--;		/* Exclude trailing whitespace.  */
 
 	      /* Expand and/or allocate storage for a slice of the name, then
-- 
2.27.0.247.g3dff7de930

[PATCH 4/4] libctf: fixes for systems on which sizeof (void *) > sizeof (long)

[Nick Alcock]

Systems like mingw64 have pointers that can only be represented by 'long
long'.  Consistently cast integers stored in pointers through uintptr_t
to cater for this.

libctf/
	* ctf-create.c (ctf_dtd_insert): Add uintptr_t casts.
	(ctf_dtd_delete): Likewise.
	(ctf_dtd_lookup): Likewise.
	(ctf_rollback): Likewise.
	* ctf-hash.c (ctf_hash_lookup_type): Likewise.
	* ctf-types.c (ctf_lookup_by_rawhash): Likewise.
---
 libctf/ctf-create.c | 16 ++++++++++------
 libctf/ctf-hash.c   |  2 +-
 libctf/ctf-types.c  |  2 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/libctf/ctf-create.c b/libctf/ctf-create.c
index 35c286c3b40..ee8757549fe 100644
--- a/libctf/ctf-create.c
+++ b/libctf/ctf-create.c
@@ -617,16 +617,19 @@ int
 ctf_dtd_insert (ctf_file_t *fp, ctf_dtdef_t *dtd, int flag, int kind)
 {
   const char *name;
-  if (ctf_dynhash_insert (fp->ctf_dthash, (void *) dtd->dtd_type, dtd) < 0)
+  if (ctf_dynhash_insert (fp->ctf_dthash, (void *) (uintptr_t) dtd->dtd_type,
+			  dtd) < 0)
     return -1;
 
   if (flag == CTF_ADD_ROOT && dtd->dtd_data.ctt_name
       && (name = ctf_strraw (fp, dtd->dtd_data.ctt_name)) != NULL)
     {
       if (ctf_dynhash_insert (ctf_name_table (fp, kind)->ctn_writable,
-			      (char *) name, (void *) dtd->dtd_type) < 0)
+			      (char *) name, (void *) (uintptr_t)
+			      dtd->dtd_type) < 0)
 	{
-	  ctf_dynhash_remove (fp->ctf_dthash, (void *) dtd->dtd_type);
+	  ctf_dynhash_remove (fp->ctf_dthash, (void *) (uintptr_t)
+			      dtd->dtd_type);
 	  return -1;
 	}
     }
@@ -642,7 +645,7 @@ ctf_dtd_delete (ctf_file_t *fp, ctf_dtdef_t *dtd)
   int name_kind = kind;
   const char *name;
 
-  ctf_dynhash_remove (fp->ctf_dthash, (void *) dtd->dtd_type);
+  ctf_dynhash_remove (fp->ctf_dthash, (void *) (uintptr_t) dtd->dtd_type);
 
   switch (kind)
     {
@@ -682,7 +685,8 @@ ctf_dtd_delete (ctf_file_t *fp, ctf_dtdef_t *dtd)
 ctf_dtdef_t *
 ctf_dtd_lookup (const ctf_file_t *fp, ctf_id_t type)
 {
-  return (ctf_dtdef_t *) ctf_dynhash_lookup (fp->ctf_dthash, (void *) type);
+  return (ctf_dtdef_t *)
+    ctf_dynhash_lookup (fp->ctf_dthash, (void *) (uintptr_t) type);
 }
 
 ctf_dtdef_t *
@@ -794,7 +798,7 @@ ctf_rollback (ctf_file_t *fp, ctf_snapshot_id_t id)
 	  ctf_str_remove_ref (fp, name, &dtd->dtd_data.ctt_name);
 	}
 
-      ctf_dynhash_remove (fp->ctf_dthash, (void *) dtd->dtd_type);
+      ctf_dynhash_remove (fp->ctf_dthash, (void *) (uintptr_t) dtd->dtd_type);
       ctf_dtd_delete (fp, dtd);
     }
 
diff --git a/libctf/ctf-hash.c b/libctf/ctf-hash.c
index 297526094cc..d6e9f5e3b6e 100644
--- a/libctf/ctf-hash.c
+++ b/libctf/ctf-hash.c
@@ -829,7 +829,7 @@ ctf_hash_lookup_type (ctf_hash_t *hp, ctf_file_t *fp __attribute__ ((__unused__)
   slot = ctf_hashtab_lookup ((struct htab *) hp, key, NO_INSERT);
 
   if (slot)
-    return (ctf_id_t) ((*slot)->value);
+    return (ctf_id_t) (uintptr_t) ((*slot)->value);
 
   return 0;
 }
diff --git a/libctf/ctf-types.c b/libctf/ctf-types.c
index ddcca66a282..4843de38d92 100644
--- a/libctf/ctf-types.c
+++ b/libctf/ctf-types.c
@@ -658,7 +658,7 @@ ctf_id_t ctf_lookup_by_rawhash (ctf_file_t *fp, ctf_names_t *np, const char *nam
   ctf_id_t id;
 
   if (fp->ctf_flags & LCTF_RDWR)
-    id = (ctf_id_t) ctf_dynhash_lookup (np->ctn_writable, name);
+    id = (ctf_id_t) (uintptr_t) ctf_dynhash_lookup (np->ctn_writable, name);
   else
     id = ctf_hash_lookup_type (np->ctn_readonly, fp, name);
   return id;
-- 
2.27.0.247.g3dff7de930

Re: [PATCH 0/4] fallout of various portability testing

[Nick Alcock]

On 22 Jul 2020, Nick Alcock verbalised:

> These patches are non-fixups that fell out of tests on bigendian Linux,

(FYI: neither this nor the fixups need review, they're just for
information: I'll be pushing later today once I've done a bit more
testing. Newly added to the test matrix: 32-bit freebsd, 32-bit cygwin,
64-bit mingw, and a cygwin64 -> linux cross.)

Re: [PATCH 00/59] Deduplicating CTF linker

[Nick Alcock]

On 1 Jul 2020, Nick Alcock via Binutils uttered the following:

> So the deduplicating CTF linker is finally ready for review.

This is pushed to master now.

If anything breaks now we have an actual testsuite, please do let me
know: there's probably a lot of untrodden snow here. (I tested a
build-and-make-check of all targets on x86_64-pc-linux-gnu, and did
manual build-and-checks on x86_64-pc-linux-gnu little- and big-endian,
FreeBSD 12, cygwin64, mingw32, mingw64, and with a cross from cygwin64
to x86_64-pc-linux-gnu. No regressions observed. But that doesn't mean
there are none...)