[PATCH 00/30] RFC: elf: glibc-hwcaps support

[PATCH 00/30] RFC: elf: glibc-hwcaps support

[Florian Weimer]

This series implements searching for alternative library implementations
under glibc-hwcaps subdirectories on the library search path.  It's
lacking tests, documentation, and a NEWS file update, but I wanted to
submit it early for discussion (and perhaps review of some of the
cleanup patches).

For background, it is best reference first the discussion about the
current hwcaps subdirectory processing:

  hwcaps subdirectory selection in the dynamic loader
  <https://sourceware.org/pipermail/libc-alpha/2020-May/113757.html>

The new approach uses a special subdirectory, called "glibc-hwcaps",
which contains nested subdirectories with all the alternative library
implementations. The glibc-hwcaps subdirectory can be present in every
subdirectory of the library search path.

ldconfig recognizes the "glibc-hwcaps" name, and it receives special
treatment if it exists: subdirectories of the directory (but the
directory itself) are scanned for libraries.  If a library is found, it
is put into the cache, and associated with the name of the subdirectory
(of glibc-hwcaps).  ldconfig does this for all subdirectories (whose
name does not begin with '.'), so it does not have to be updated if new
subdirectories are added.  There is currently no symbolic link
processing for these subdirectories.  In the future, they might store
multiple implementations with different GNU property notes, and we might
want to select those through the cache as well.  (I'm open to changing
this part to align more with traditional cache processing.)

At run time, the dynamic loader has a built-in list of HWCAP
subdirectory names (which are different from the legacy HWCAP
directories).  The new “ld.so --help” option shows these subdirectory
names.  For LD_LIBRARY_PATH processing, ld.so simply attempts to open
libraries under their soname in the active/supported glibc-hwcaps
subdirectories.  (This part is quite similar to the original HWCAP
subdirectory handling, except that the new subdirectories do not nest.)
For ld.so.cache processing, ld.so sorts the active/supported HWCAP
subdirectory names and merges this list against the list of subdirectory
names as stored in a new ld.so.cache extension section.  Each
glibc-hwcaps library listed in the cache has a HWCAP field that contains
an index into the HWCAP subdirectory array of the cache file.  (A
previously unused HWCAP bit is set so that these entries are ignored by
existing dynamic linkers.)  The dynamic linker loads the library with
has the lowest positive preference value (essentially from the
subdirectory that comes first in its consolidated HWCAP name list).

There are new ld.so options, --glibc-hwcaps-prepend and
--glibc-hwcaps-mask, to change the list of glibc-hwcaps subdirectories
searched.  Due to the way ldconfig has been modified, it is possible to
pick up completely new subdirectories (not known to the implementation)
using --glibc-hwcaps-prepend even if ld.so.cache is used.

With these changes, on a current x86-64 machine (with AVX2-level CPU
features), you can drop a shared object into the directory

  /usr/lib64/glibc-hwcaps/x86-102

and the dynamic linker will load it, either via ld.so.cache or
LD_LIBRARY_PATH processing.

Under strace, it look like this if the cache is used:

| openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
| openat(AT_FDCWD, "/lib64/glibc-hwcaps/x86-102/libz.so.1", O_RDONLY|O_CLOEXEC) = 3

Diagnostics output from “ld.so --help” for this machine is this:

| Shared library search path:
|   (libraries located via /etc/ld.so.cache)
|   /lib64 (system search path)
|   /usr/lib64 (system search path)
| 
| Subdirectories of glibc-hwcaps directories, in priority order:
|   x86-103
|   x86-102 (supported, searched)
|   x86-101 (supported, searched)
|   x86-100 (supported, searched)
| 
| Legacy HWCAP subdirectories under library search path directories:
|   haswell (AT_PLATFORM)
|   tls (supported, searched)
|   avx512_1
|   x86_64 (supported, searched)

I've also included support for a "power9" subdirectory on powerpc64le,
some subdirectories on s390x, and "atomics" on aarch64.  The exact
definitions of the directories are still subject to change.  They are
there to that there is at least something new to be searched.  For the
x86-64 part, I'm going to start a separate, cross-project discussion
about the desired CPU feature flags for each micro-architecture baseline
level.

The series does not yet check if the glibc-hwcaps directory itself
exists.  If the directory does not exist, we can skip searching all its
subdirectories.  This is another motivation for having the magic
glibc-hwcaps subdirectory.  But I have not implemented this yet because
it tends to hide the search activity from strace.

Patches up to (but excluding) “elf: Add glibc-hwcaps support for
LD_LIBRARY_PATH” are more or less cleanups, and so are these patches:

  elf: Unify old and new format cache handling code in ld.so
  elf: Implement a string table for ldconfig, with tail merging
  elf: Implement tail merging of strings in ldconfig
  elf: In ldconfig, extract the new_sub_entry function from search_dir

These cleanups could be reviewed and installed separately.

Thanks,
Florian

Florian Weimer (30):
  elf: Include <stdbool.h> in <dl-tunables.h> because bool is used
  elf: Include <stddef.h> (for size_t), <sys/stat.h> in <ldconfig.h>
  elf: Do not search HWCAP subdirectories in statically linked binaries
  elf: Implement __rtld_malloc_is_full
  elf: Implement _dl_write
  elf: Extract command-line/environment variables state from rtld.c
  elf: Move ld.so error/help output to _dl_usage
  elf: Record whether paths come from LD_LIBRARY_PATH or --library-path
  elf: Implement ld.so --help
  elf: Implement ld.so --version
  scripts/update-copyrights: Update csu/version.c, elf/dl-usage.c
  elf: Use the term "program interpreter" in the ld.so help message
  elf: Print the full name of the dynamic loader in the ld.so help
    message
  elf: Make __rtld_env_path_list and __rtld_search_dirs global variables
  elf: Add library search path information to ld.so --help
  elf: Enhance ld.so --help to print HWCAP subdirectories
  elf: Do not pass GLRO(dl_platform), GLRO(dl_platformlen) to
    _dl_important_hwcaps
  elf: Add glibc-hwcaps support for LD_LIBRARY_PATH
  x86_64: Add glibc-hwcaps support
  powerpc64le: Add glibc-hwcaps support
  s390x: Add Add glibc-hwcaps support
  aarch64: Add glibc-hwcaps support
  elf: Add endianness markup to ld.so.cache
  elf: Add extension mechanism to ld.so.cache
  elf: Unify old and new format cache handling code in ld.so
  elf: Implement a string table for ldconfig, with tail merging
  elf: Implement tail merging of strings in ldconfig
  elf: In ldconfig, extract the new_sub_entry function from search_dir
  elf: Process glibc-hwcaps subdirectories in ldconfig
  elf: Add glibc-hwcaps subdirectory support to ld.so cache processing

 NEWS                                          |   4 +
 elf/Makefile                                  |  22 +-
 elf/cache.c                                   | 400 +++++++++++++--
 elf/dl-cache.c                                | 483 ++++++++++++------
 elf/dl-hwcaps-subdirs.c                       |  29 ++
 elf/dl-hwcaps.c                               | 225 +++++++-
 elf/dl-hwcaps.h                               | 102 ++++
 elf/dl-hwcaps_split.c                         |  77 +++
 elf/dl-load.c                                 |  75 +--
 elf/dl-main.h                                 | 120 +++++
 elf/dl-minimal.c                              |   8 +
 elf/dl-support.c                              |   5 +-
 elf/dl-tunables.h                             |   2 +
 elf/dl-usage.c                                | 267 ++++++++++
 elf/dl-write.c                                |  56 ++
 elf/ldconfig.c                                | 187 +++++--
 elf/rtld.c                                    | 248 ++++-----
 elf/stringtable.c                             | 201 ++++++++
 elf/stringtable.h                             |  61 +++
 elf/stringtable_free.c                        |  32 ++
 elf/tst-dl-hwcaps_split.c                     | 139 +++++
 elf/tst-stringtable.c                         | 140 +++++
 include/link.h                                |   4 +
 include/rtld-malloc.h                         |   4 +
 scripts/update-copyrights                     |   6 +
 sysdeps/aarch64/dl-hwcaps-subdirs.c           |  31 ++
 sysdeps/generic/dl-cache.h                    | 232 ++++++++-
 sysdeps/generic/ldconfig.h                    |  20 +-
 sysdeps/generic/ldsodefs.h                    |  34 +-
 .../powerpc/powerpc64/le/dl-hwcaps-subdirs.c  |  31 ++
 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c      |  54 ++
 sysdeps/unix/sysv/linux/dl-write.c            |  30 ++
 sysdeps/x86_64/dl-hwcaps-subdirs.c            |  73 +++
 33 files changed, 2980 insertions(+), 422 deletions(-)
 create mode 100644 elf/dl-hwcaps-subdirs.c
 create mode 100644 elf/dl-hwcaps_split.c
 create mode 100644 elf/dl-main.h
 create mode 100644 elf/dl-usage.c
 create mode 100644 elf/dl-write.c
 create mode 100644 elf/stringtable.c
 create mode 100644 elf/stringtable.h
 create mode 100644 elf/stringtable_free.c
 create mode 100644 elf/tst-dl-hwcaps_split.c
 create mode 100644 elf/tst-stringtable.c
 create mode 100644 sysdeps/aarch64/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c
 create mode 100644 sysdeps/unix/sysv/linux/dl-write.c
 create mode 100644 sysdeps/x86_64/dl-hwcaps-subdirs.c

-- 
2.25.4

[PATCH 01/30] elf: Include <stdbool.h> in <dl-tunables.h> because bool is used

[Florian Weimer]

---
 elf/dl-tunables.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
index 969e50327b..f05eb50c2f 100644
--- a/elf/dl-tunables.h
+++ b/elf/dl-tunables.h
@@ -21,6 +21,8 @@
 #ifndef _TUNABLES_H_
 #define _TUNABLES_H_
 
+#include <stdbool.h>
+
 #if !HAVE_TUNABLES
 static inline void
 __always_inline
-- 
2.25.4

[PATCH 02/30] elf: Include <stddef.h> (for size_t), <sys/stat.h> in <ldconfig.h>

[Florian Weimer]

---
 sysdeps/generic/ldconfig.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sysdeps/generic/ldconfig.h b/sysdeps/generic/ldconfig.h
index a8a2be7856..b64aab0064 100644
--- a/sysdeps/generic/ldconfig.h
+++ b/sysdeps/generic/ldconfig.h
@@ -19,7 +19,9 @@
 #ifndef _LDCONFIG_H
 #define _LDCONFIG_H
 
+#include <stddef.h>
 #include <stdint.h>
+#include <sys/stat.h>
 
 #define FLAG_ANY			-1
 #define FLAG_TYPE_MASK			0x00ff
-- 
2.25.4

[PATCH 03/30] elf: Do not search HWCAP subdirectories in statically linked binaries

[Florian Weimer]

This functionality does not seem to be useful since static dlopen
is mostly used for iconv/character set conversion and NSS support.
gconv modules are loaded with full paths anyway, so that the
HWCAP subdirectory logic does not apply.
---
 NEWS          |  4 ++++
 elf/Makefile  |  4 ++--
 elf/dl-load.c | 14 ++++++++++++++
 3 files changed, 20 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index a660fc59a8..032255d58e 100644
--- a/NEWS
+++ b/NEWS
@@ -51,6 +51,10 @@ Deprecated and removed features, and other changes affecting compatibility:
 * ldconfig now defaults to the new format for ld.so.cache. glibc has
   already supported this format for almost 20 years.
 
+* When dlopen is used in statically linked programs, alternative library
+  implementations from HWCAP subdirectories are no longer loaded.
+  Instead, the default implementation is used.
+
 Changes to build and runtime requirements:
 
 * powerpc64le requires GCC 7.4 or newer.  This is required for supporting
diff --git a/elf/Makefile b/elf/Makefile
index 6fe1df90bb..1293de5b5e 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -29,7 +29,7 @@ routines	= $(all-dl-routines) dl-support dl-iteratephdr \
 
 # The core dynamic linking functions are in libc for the static and
 # profiled libraries.
-dl-routines	= $(addprefix dl-,load lookup object reloc deps hwcaps \
+dl-routines	= $(addprefix dl-,load lookup object reloc deps \
 				  runtime init fini debug misc \
 				  version profile tls origin scope \
 				  execstack open close trampoline \
@@ -59,7 +59,7 @@ elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \
 # ld.so uses those routines, plus some special stuff for being the program
 # interpreter and operating independent of libc.
 rtld-routines	= rtld $(all-dl-routines) dl-sysdep dl-environ dl-minimal \
-  dl-error-minimal dl-conflict
+  dl-error-minimal dl-conflict dl-hwcaps
 all-rtld-routines = $(rtld-routines) $(sysdep-rtld-routines)
 
 CFLAGS-dl-runtime.c += -fexceptions -fasynchronous-unwind-tables
diff --git a/elf/dl-load.c b/elf/dl-load.c
index 06f2ba7264..34d3b02a95 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -101,9 +101,13 @@ int __stack_prot attribute_hidden attribute_relro
 static struct r_search_path_struct env_path_list attribute_relro;
 
 /* List of the hardware capabilities we might end up using.  */
+#ifdef SHARED
 static const struct r_strlenpair *capstr attribute_relro;
 static size_t ncapstr attribute_relro;
 static size_t max_capstrlen attribute_relro;
+#else
+enum { ncapstr = 1, max_capstrlen = 0 };
+#endif
 
 
 /* Get the generated information about the trusted directories.  Use
@@ -691,9 +695,11 @@ _dl_init_paths (const char *llp)
   /* Fill in the information about the application's RPATH and the
      directories addressed by the LD_LIBRARY_PATH environment variable.  */
 
+#ifdef SHARED
   /* Get the capabilities.  */
   capstr = _dl_important_hwcaps (GLRO(dl_platform), GLRO(dl_platformlen),
 				 &ncapstr, &max_capstrlen);
+#endif
 
   /* First set up the rest of the default search directory entries.  */
   aelem = rtld_search_dirs.dirs = (struct r_search_path_elem **)
@@ -1443,11 +1449,15 @@ print_search_path (struct r_search_path_elem **list,
       for (cnt = 0; cnt < ncapstr; ++cnt)
 	if ((*list)->status[cnt] != nonexisting)
 	  {
+#ifdef SHARED
 	    char *cp = __mempcpy (endp, capstr[cnt].str, capstr[cnt].len);
 	    if (cp == buf || (cp == buf + 1 && buf[0] == '/'))
 	      cp[0] = '\0';
 	    else
 	      cp[-1] = '\0';
+#else
+	    *endp = '\0';
+#endif
 
 	    _dl_debug_printf_c (first ? "%s" : ":%s", buf);
 	    first = 0;
@@ -1808,11 +1818,15 @@ open_path (const char *name, size_t namelen, int mode,
 	  if (this_dir->status[cnt] == nonexisting)
 	    continue;
 
+#ifdef SHARED
 	  buflen =
 	    ((char *) __mempcpy (__mempcpy (edp, capstr[cnt].str,
 					    capstr[cnt].len),
 				 name, namelen)
 	     - buf);
+#else
+	  buflen = (char *) __mempcpy (edp, name, namelen) - buf;
+#endif
 
 	  /* Print name we try if this is wanted.  */
 	  if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))
-- 
2.25.4

[PATCH 04/30] elf: Implement __rtld_malloc_is_full

[Florian Weimer]

In some cases, it is difficult to determine the kind of malloc
based on the execution context, so a function to determine that
is helpful.
---
 elf/dl-minimal.c      | 8 ++++++++
 include/rtld-malloc.h | 4 ++++
 2 files changed, 12 insertions(+)

diff --git a/elf/dl-minimal.c b/elf/dl-minimal.c
index 7c64e24c87..dc79f02458 100644
--- a/elf/dl-minimal.c
+++ b/elf/dl-minimal.c
@@ -59,6 +59,14 @@ __rtld_malloc_init_stubs (void)
   __rtld_realloc = &rtld_realloc;
 }
 
+bool
+__rtld_malloc_is_full (void)
+{
+  /* The caller assumes that there is an active malloc.  */
+  assert (__rtld_malloc != NULL);
+  return __rtld_malloc != &rtld_malloc;
+}
+
 /* Lookup NAME at VERSION in the scope of MATCH.  */
 static void *
 lookup_malloc_symbol (struct link_map *main_map, const char *name,
diff --git a/include/rtld-malloc.h b/include/rtld-malloc.h
index b026a3270c..54f53f7888 100644
--- a/include/rtld-malloc.h
+++ b/include/rtld-malloc.h
@@ -66,6 +66,10 @@ realloc (void *ptr, size_t size)
    implementation.  */
 void __rtld_malloc_init_stubs (void) attribute_hidden;
 
+/* Return false if the active malloc is the ld.so minimal malloc, true
+   if it is the full implementation from libc.so.  */
+_Bool __rtld_malloc_is_full (void) attribute_hidden;
+
 /* Called shortly before the final self-relocation (when RELRO
    variables are still writable) to activate the real malloc
    implementation.  MAIN_MAP is the link map of the executable.  */
-- 
2.25.4

[PATCH 05/30] elf: Implement _dl_write

[Florian Weimer]

The generic version is parallel to _dl_writev.  It cannot use
_dl_writev directly because the errno value needs to be obtained
under a lock.
---
 elf/Makefile                       |  2 +-
 elf/dl-write.c                     | 56 ++++++++++++++++++++++++++++++
 sysdeps/generic/ldsodefs.h         |  6 ++++
 sysdeps/unix/sysv/linux/dl-write.c | 30 ++++++++++++++++
 4 files changed, 93 insertions(+), 1 deletion(-)
 create mode 100644 elf/dl-write.c
 create mode 100644 sysdeps/unix/sysv/linux/dl-write.c

diff --git a/elf/Makefile b/elf/Makefile
index 1293de5b5e..b56fe0c0b7 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -34,7 +34,7 @@ dl-routines	= $(addprefix dl-,load lookup object reloc deps \
 				  version profile tls origin scope \
 				  execstack open close trampoline \
 				  exception sort-maps lookup-direct \
-				  call-libc-early-init)
+				  call-libc-early-init write)
 ifeq (yes,$(use-ldconfig))
 dl-routines += dl-cache
 endif
diff --git a/elf/dl-write.c b/elf/dl-write.c
new file mode 100644
index 0000000000..7350aff003
--- /dev/null
+++ b/elf/dl-write.c
@@ -0,0 +1,56 @@
+/* Implementation of the _dl_write function.  Generic version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <errno.h>
+#include <ldsodefs.h>
+#include <libc-lock.h>
+#include <sys/uio.h>
+
+ssize_t
+_dl_write (int fd, const void *buffer, size_t length)
+{
+  struct iovec iov = { .iov_base = (void *) buffer, .iov_len = length };
+  ssize_t ret;
+
+#if RTLD_PRIVATE_ERRNO
+  /* We have to take this lock just to be sure we don't clobber the private
+     errno when it's being used by another thread that cares about it.
+     Yet we must be sure not to try calling the lock functions before
+     the thread library is fully initialized.  */
+  if (__glibc_unlikely (_dl_starting_up))
+    {
+      ret = __writev (fd, &iov, 1);
+      if (ret < 0)
+        ret = -errno;
+    }
+  else
+    {
+      __rtld_lock_lock_recursive (GL(dl_load_lock));
+      __writev (fd, &iov, 1);
+      if (ret < 0)
+        ret = -errno;
+      __rtld_lock_unlock_recursive (GL(dl_load_lock));
+    }
+#else
+  ret = __writev (fd, &iov, 1);
+  if (ret < 0)
+    ret = -errno;
+#endif
+
+  return ret;
+}
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index d08b97a5ef..42fe912f47 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -749,6 +749,12 @@ _dl_dprintf (int fd, const char *fmt, ...)
 }
 #endif
 
+/* Write LENGTH bytes at BUFFER to FD, like write.  Returns the number
+   of bytes written on success, or a negative error constant on
+   failure.  */
+ssize_t _dl_write (int fd, const void *buffer, size_t length)
+  attribute_hidden;
+
 /* Write a message on the specified descriptor standard output.  The
    parameters are interpreted as for a `printf' call.  */
 void _dl_printf (const char *fmt, ...)
diff --git a/sysdeps/unix/sysv/linux/dl-write.c b/sysdeps/unix/sysv/linux/dl-write.c
new file mode 100644
index 0000000000..1c6298fb41
--- /dev/null
+++ b/sysdeps/unix/sysv/linux/dl-write.c
@@ -0,0 +1,30 @@
+/* Implementation of the _dl_write function.  Linux version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <sysdep.h>
+#include <unistd.h>
+#include <ldsodefs.h>
+
+ssize_t
+_dl_write (int fd, const void *buffer, size_t length)
+{
+  long int r = INTERNAL_SYSCALL_CALL (write, fd, buffer, length);
+  if (INTERNAL_SYSCALL_ERROR_P (r))
+    r = - INTERNAL_SYSCALL_ERRNO (r);
+  return r;
+}
-- 
2.25.4

[PATCH 06/30] elf: Extract command-line/environment variables state from rtld.c

[Florian Weimer]

Introduce struct dl_main_state and move it to <dl-main.h>.

This avoids the need for putting state that is only needed during
startup into the ld.so data segment.
---
 elf/dl-main.h |  95 +++++++++++++++++++++++++++++++
 elf/rtld.c    | 153 +++++++++++++++++++-------------------------------
 2 files changed, 153 insertions(+), 95 deletions(-)
 create mode 100644 elf/dl-main.h

diff --git a/elf/dl-main.h b/elf/dl-main.h
new file mode 100644
index 0000000000..ad9250171f
--- /dev/null
+++ b/elf/dl-main.h
@@ -0,0 +1,95 @@
+/* Information collection during ld.so startup.
+   Copyright (C) 1995-2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#ifndef _DL_MAIN
+#define _DL_MAIN
+
+#include <limits.h>
+
+/* Length limits for names and paths, to protect the dynamic linker,
+   particularly when __libc_enable_secure is active.  */
+#ifdef NAME_MAX
+# define SECURE_NAME_LIMIT NAME_MAX
+#else
+# define SECURE_NAME_LIMIT 255
+#endif
+#ifdef PATH_MAX
+# define SECURE_PATH_LIMIT PATH_MAX
+#else
+# define SECURE_PATH_LIMIT 1024
+#endif
+
+/* Strings containing colon-separated lists of audit modules.  */
+struct audit_list
+{
+  /* Array of strings containing colon-separated path lists.  Each
+     audit module needs its own namespace, so pre-allocate the largest
+     possible list.  */
+  const char *audit_strings[DL_NNS];
+
+  /* Number of entries added to audit_strings.  */
+  size_t length;
+
+  /* Index into the audit_strings array (for the iteration phase).  */
+  size_t current_index;
+
+  /* Tail of audit_strings[current_index] which still needs
+     processing.  */
+  const char *current_tail;
+
+  /* Scratch buffer for returning a name which is part of the strings
+     in audit_strings.  */
+  char fname[SECURE_NAME_LIMIT];
+};
+
+/* This is a list of all the modes the dynamic loader can be in.  */
+enum mode { normal, list, verify, trace };
+
+/* Aggregated state information extracted from environment variables
+   and the ld.so command line.  */
+struct dl_main_state
+{
+  struct audit_list audit_list;
+
+  /* The library search path.  */
+  const char *library_path;
+
+  /* The list preloaded objects from LD_PRELOAD.  */
+  const char *preloadlist;
+
+  /* The preload list passed as a command argument.  */
+  const char *preloadarg;
+
+  enum mode mode;
+
+  /* True if any of the debugging options is enabled.  */
+  bool any_debug;
+
+  /* True if information about versions has to be printed.  */
+  bool version_info;
+};
+
+/* Helper function to invoke _dl_init_paths with the right arguments
+   from *STATE.  */
+static inline void
+call_init_paths (const struct dl_main_state *state)
+{
+  _dl_init_paths (state->library_path);
+}
+
+#endif /* _DL_MAIN */
diff --git a/elf/rtld.c b/elf/rtld.c
index 882b070cc0..e3556eabc4 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -47,6 +47,7 @@
 #include <not-cancel.h>
 #include <array_length.h>
 #include <libc-early-init.h>
+#include <dl-main.h>
 
 #include <assert.h>
 
@@ -111,42 +112,6 @@ static void print_missing_version (int errcode, const char *objname,
 /* Print the various times we collected.  */
 static void print_statistics (const hp_timing_t *total_timep);
 
-/* Length limits for names and paths, to protect the dynamic linker,
-   particularly when __libc_enable_secure is active.  */
-#ifdef NAME_MAX
-# define SECURE_NAME_LIMIT NAME_MAX
-#else
-# define SECURE_NAME_LIMIT 255
-#endif
-#ifdef PATH_MAX
-# define SECURE_PATH_LIMIT PATH_MAX
-#else
-# define SECURE_PATH_LIMIT 1024
-#endif
-
-/* Strings containing colon-separated lists of audit modules.  */
-struct audit_list
-{
-  /* Array of strings containing colon-separated path lists.  Each
-     audit module needs its own namespace, so pre-allocate the largest
-     possible list.  */
-  const char *audit_strings[DL_NNS];
-
-  /* Number of entries added to audit_strings.  */
-  size_t length;
-
-  /* Index into the audit_strings array (for the iteration phase).  */
-  size_t current_index;
-
-  /* Tail of audit_strings[current_index] which still needs
-     processing.  */
-  const char *current_tail;
-
-  /* Scratch buffer for returning a name which is part of the strings
-     in audit_strings.  */
-  char fname[SECURE_NAME_LIMIT];
-};
-
 /* Creates an empty audit list.  */
 static void audit_list_init (struct audit_list *);
 
@@ -167,13 +132,13 @@ static void audit_list_add_dynamic_tag (struct audit_list *,
    audit_list_add_dynamic_tags calls.  */
 static const char *audit_list_next (struct audit_list *);
 
-/* This is a list of all the modes the dynamic loader can be in.  */
-enum mode { normal, list, verify, trace };
+/* Initialize *STATE with the defaults.  */
+static void dl_main_state_init (struct dl_main_state *state);
 
 /* Process all environments variables the dynamic linker must recognize.
    Since all of them start with `LD_' we are a bit smarter while finding
    all the entries.  */
-static void process_envvars (enum mode *modep, struct audit_list *);
+static void process_envvars (struct dl_main_state *state);
 
 #ifdef DL_ARGV_NOT_RELRO
 int _dl_argc attribute_hidden;
@@ -299,6 +264,18 @@ audit_list_next (struct audit_list *list)
     }
 }
 
+static void
+dl_main_state_init (struct dl_main_state *state)
+{
+  audit_list_init (&state->audit_list);
+  state->library_path = NULL;
+  state->preloadlist = NULL;
+  state->preloadarg = NULL;
+  state->mode = normal;
+  state->any_debug = false;
+  state->version_info = false;
+}
+
 #ifndef HAVE_INLINED_SYSCALLS
 /* Set nonzero during loading and initialization of executable and
    libraries, cleared before the executable's entry point runs.  This
@@ -880,15 +857,6 @@ security_init (void)
 
 #include <setup-vdso.h>
 
-/* The library search path.  */
-static const char *library_path attribute_relro;
-/* The list preloaded objects.  */
-static const char *preloadlist attribute_relro;
-/* Nonzero if information about versions has to be printed.  */
-static int version_info attribute_relro;
-/* The preload list passed as a command argument.  */
-static const char *preloadarg attribute_relro;
-
 /* The LD_PRELOAD environment variable gives list of libraries
    separated by white space or colons that are loaded before the
    executable's dependencies and prepended to the global scope list.
@@ -1130,7 +1098,6 @@ dl_main (const ElfW(Phdr) *phdr,
 	 ElfW(auxv_t) *auxv)
 {
   const ElfW(Phdr) *ph;
-  enum mode mode;
   struct link_map *main_map;
   size_t file_size;
   char *file;
@@ -1140,8 +1107,8 @@ dl_main (const ElfW(Phdr) *phdr,
   bool rtld_is_main = false;
   void *tcbp = NULL;
 
-  struct audit_list audit_list;
-  audit_list_init (&audit_list);
+  struct dl_main_state state;
+  dl_main_state_init (&state);
 
   GL(dl_init_static_tls) = &_dl_nothread_init_static_tls;
 
@@ -1156,7 +1123,7 @@ dl_main (const ElfW(Phdr) *phdr,
   GL(dl_make_stack_executable_hook) = &_dl_make_stack_executable;
 
   /* Process the environment variable which control the behaviour.  */
-  process_envvars (&mode, &audit_list);
+  process_envvars (&state);
 
 #ifndef HAVE_INLINED_SYSCALLS
   /* Set up a flag which tells we are just starting.  */
@@ -1188,7 +1155,7 @@ dl_main (const ElfW(Phdr) *phdr,
       while (_dl_argc > 1)
 	if (! strcmp (_dl_argv[1], "--list"))
 	  {
-	    mode = list;
+	    state.mode = list;
 	    GLRO(dl_lazy) = -1;	/* This means do no dependency analysis.  */
 
 	    ++_dl_skip_args;
@@ -1197,7 +1164,7 @@ dl_main (const ElfW(Phdr) *phdr,
 	  }
 	else if (! strcmp (_dl_argv[1], "--verify"))
 	  {
-	    mode = verify;
+	    state.mode = verify;
 
 	    ++_dl_skip_args;
 	    --_dl_argc;
@@ -1213,7 +1180,7 @@ dl_main (const ElfW(Phdr) *phdr,
 	else if (! strcmp (_dl_argv[1], "--library-path")
 		 && _dl_argc > 2)
 	  {
-	    library_path = _dl_argv[2];
+	    state.library_path = _dl_argv[2];
 
 	    _dl_skip_args += 2;
 	    _dl_argc -= 2;
@@ -1230,7 +1197,7 @@ dl_main (const ElfW(Phdr) *phdr,
 	  }
 	else if (! strcmp (_dl_argv[1], "--audit") && _dl_argc > 2)
 	  {
-	    audit_list_add_string (&audit_list, _dl_argv[2]);
+	    audit_list_add_string (&state.audit_list, _dl_argv[2]);
 
 	    _dl_skip_args += 2;
 	    _dl_argc -= 2;
@@ -1238,7 +1205,7 @@ dl_main (const ElfW(Phdr) *phdr,
 	  }
 	else if (! strcmp (_dl_argv[1], "--preload") && _dl_argc > 2)
 	  {
-	    preloadarg = _dl_argv[2];
+	    state.preloadarg = _dl_argv[2];
 	    _dl_skip_args += 2;
 	    _dl_argc -= 2;
 	    _dl_argv += 2;
@@ -1297,7 +1264,7 @@ of this helper program; chances are you did not intend to run this program.\n\
 	    break;
 	  }
 
-      if (__builtin_expect (mode, normal) == verify)
+      if (__builtin_expect (state.mode, normal) == verify)
 	{
 	  const char *objname;
 	  const char *err_str = NULL;
@@ -1326,7 +1293,7 @@ of this helper program; chances are you did not intend to run this program.\n\
       /* Now the map for the main executable is available.  */
       main_map = GL(dl_ns)[LM_ID_BASE]._ns_loaded;
 
-      if (__builtin_expect (mode, normal) == normal
+      if (__builtin_expect (state.mode, normal) == normal
 	  && GL(dl_rtld_map).l_info[DT_SONAME] != NULL
 	  && main_map->l_info[DT_SONAME] != NULL
 	  && strcmp ((const char *) D_PTR (&GL(dl_rtld_map), l_info[DT_STRTAB])
@@ -1563,7 +1530,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
       _dl_setup_hash (main_map);
     }
 
-  if (__builtin_expect (mode, normal) == verify)
+  if (__builtin_expect (state.mode, normal) == verify)
     {
       /* We were called just to verify that this is a dynamic
 	 executable using us as the program interpreter.  Exit with an
@@ -1593,7 +1560,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
 
   /* Initialize the data structures for the search paths for shared
      objects.  */
-  _dl_init_paths (library_path);
+  call_init_paths (&state);
 
   /* Initialize _r_debug.  */
   struct r_debug *r = _dl_debug_initialize (GL(dl_rtld_map).l_addr,
@@ -1658,12 +1625,12 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
     /* Assign a module ID.  Do this before loading any audit modules.  */
     GL(dl_rtld_map).l_tls_modid = _dl_next_tls_modid ();
 
-  audit_list_add_dynamic_tag (&audit_list, main_map, DT_AUDIT);
-  audit_list_add_dynamic_tag (&audit_list, main_map, DT_DEPAUDIT);
+  audit_list_add_dynamic_tag (&state.audit_list, main_map, DT_AUDIT);
+  audit_list_add_dynamic_tag (&state.audit_list, main_map, DT_DEPAUDIT);
 
   /* If we have auditing DSOs to load, do it now.  */
   bool need_security_init = true;
-  if (audit_list.length > 0)
+  if (state.audit_list.length > 0)
     {
       /* Since we start using the auditing DSOs right away we need to
 	 initialize the data structures now.  */
@@ -1676,7 +1643,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
       security_init ();
       need_security_init = false;
 
-      load_audit_modules (main_map, &audit_list);
+      load_audit_modules (main_map, &state.audit_list);
     }
 
   /* Keep track of the currently loaded modules to count how many
@@ -1727,19 +1694,21 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
   struct link_map **preloads = NULL;
   unsigned int npreloads = 0;
 
-  if (__glibc_unlikely (preloadlist != NULL))
+  if (__glibc_unlikely (state.preloadlist != NULL))
     {
       RTLD_TIMING_VAR (start);
       rtld_timer_start (&start);
-      npreloads += handle_preload_list (preloadlist, main_map, "LD_PRELOAD");
+      npreloads += handle_preload_list (state.preloadlist, main_map,
+					"LD_PRELOAD");
       rtld_timer_accum (&load_time, start);
     }
 
-  if (__glibc_unlikely (preloadarg != NULL))
+  if (__glibc_unlikely (state.preloadarg != NULL))
     {
       RTLD_TIMING_VAR (start);
       rtld_timer_start (&start);
-      npreloads += handle_preload_list (preloadarg, main_map, "--preload");
+      npreloads += handle_preload_list (state.preloadarg, main_map,
+					"--preload");
       rtld_timer_accum (&load_time, start);
     }
 
@@ -1846,7 +1815,8 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
   {
     RTLD_TIMING_VAR (start);
     rtld_timer_start (&start);
-    _dl_map_object_deps (main_map, preloads, npreloads, mode == trace, 0);
+    _dl_map_object_deps (main_map, preloads, npreloads,
+			 state.mode == trace, 0);
     rtld_timer_accum (&load_time, start);
   }
 
@@ -1873,7 +1843,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
       rtld_multiple_ref = true;
 
       GL(dl_rtld_map).l_prev = main_map->l_searchlist.r_list[i - 1];
-      if (__builtin_expect (mode, normal) == normal)
+      if (__builtin_expect (state.mode, normal) == normal)
 	{
 	  GL(dl_rtld_map).l_next = (i + 1 < main_map->l_searchlist.r_nlist
 				    ? main_map->l_searchlist.r_list[i + 1]
@@ -1906,8 +1876,8 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
      versions we need.  */
   {
     struct version_check_args args;
-    args.doexit = mode == normal;
-    args.dotrace = mode == trace;
+    args.doexit = state.mode == normal;
+    args.dotrace = state.mode == trace;
     _dl_receive_error (print_missing_version, version_check_doit, &args);
   }
 
@@ -1927,7 +1897,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
        earlier.  */
     security_init ();
 
-  if (__builtin_expect (mode, normal) != normal)
+  if (__builtin_expect (state.mode, normal) != normal)
     {
       /* We were run just to list the shared libraries.  It is
 	 important that we do this before real relocation, because the
@@ -2029,7 +1999,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
 			  (size_t) l->l_map_start);
 	}
 
-      if (__builtin_expect (mode, trace) != trace)
+      if (__builtin_expect (state.mode, trace) != trace)
 	for (i = 1; i < (unsigned int) _dl_argc; ++i)
 	  {
 	    const ElfW(Sym) *ref = NULL;
@@ -2083,7 +2053,7 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
 		}
 	    }
 #define VERNEEDTAG (DT_NUM + DT_THISPROCNUM + DT_VERSIONTAGIDX (DT_VERNEED))
-	  if (version_info)
+	  if (state.version_info)
 	    {
 	      /* Print more information.  This means here, print information
 		 about the versions needed.  */
@@ -2445,13 +2415,10 @@ print_missing_version (int errcode __attribute__ ((unused)),
 		    objname, errstring);
 }
 \f
-/* Nonzero if any of the debugging options is enabled.  */
-static int any_debug attribute_relro;
-
 /* Process the string given as the parameter which explains which debugging
    options are enabled.  */
 static void
-process_dl_debug (const char *dl_debug)
+process_dl_debug (struct dl_main_state *state, const char *dl_debug)
 {
   /* When adding new entries make sure that the maximal length of a name
      is correctly handled in the LD_DEBUG_HELP code below.  */
@@ -2508,7 +2475,7 @@ process_dl_debug (const char *dl_debug)
 		&& memcmp (dl_debug, debopts[cnt].name, len) == 0)
 	      {
 		GLRO(dl_debug_mask) |= debopts[cnt].mask;
-		any_debug = 1;
+		state->any_debug = true;
 		break;
 	      }
 
@@ -2562,11 +2529,10 @@ extern char **_environ attribute_hidden;
 
 
 static void
-process_envvars (enum mode *modep, struct audit_list *audit_list)
+process_envvars (struct dl_main_state *state)
 {
   char **runp = _environ;
   char *envline;
-  enum mode mode = normal;
   char *debug_output = NULL;
 
   /* This is the default place for profiling data file.  */
@@ -2598,25 +2564,25 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	  /* Debugging of the dynamic linker?  */
 	  if (memcmp (envline, "DEBUG", 5) == 0)
 	    {
-	      process_dl_debug (&envline[6]);
+	      process_dl_debug (state, &envline[6]);
 	      break;
 	    }
 	  if (memcmp (envline, "AUDIT", 5) == 0)
-	    audit_list_add_string (audit_list, &envline[6]);
+	    audit_list_add_string (&state->audit_list, &envline[6]);
 	  break;
 
 	case 7:
 	  /* Print information about versions.  */
 	  if (memcmp (envline, "VERBOSE", 7) == 0)
 	    {
-	      version_info = envline[8] != '\0';
+	      state->version_info = envline[8] != '\0';
 	      break;
 	    }
 
 	  /* List of objects to be preloaded.  */
 	  if (memcmp (envline, "PRELOAD", 7) == 0)
 	    {
-	      preloadlist = &envline[8];
+	      state->preloadlist = &envline[8];
 	      break;
 	    }
 
@@ -2665,7 +2631,7 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	  if (!__libc_enable_secure
 	      && memcmp (envline, "LIBRARY_PATH", 12) == 0)
 	    {
-	      library_path = &envline[13];
+	      state->library_path = &envline[13];
 	      break;
 	    }
 
@@ -2707,7 +2673,7 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	  /* The mode of the dynamic linker can be set.  */
 	  if (memcmp (envline, "TRACE_PRELINKING", 16) == 0)
 	    {
-	      mode = trace;
+	      state->mode = trace;
 	      GLRO(dl_verbose) = 1;
 	      GLRO(dl_debug_mask) |= DL_DEBUG_PRELINK;
 	      GLRO(dl_trace_prelink) = &envline[17];
@@ -2717,7 +2683,7 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	case 20:
 	  /* The mode of the dynamic linker can be set.  */
 	  if (memcmp (envline, "TRACE_LOADED_OBJECTS", 20) == 0)
-	    mode = trace;
+	    state->mode = trace;
 	  break;
 
 	  /* We might have some extra environment variable to handle.  This
@@ -2730,9 +2696,6 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	}
     }
 
-  /* The caller wants this information.  */
-  *modep = mode;
-
   /* Extra security for SUID binaries.  Remove all dangerous environment
      variables.  */
   if (__builtin_expect (__libc_enable_secure, 0))
@@ -2761,13 +2724,13 @@ process_envvars (enum mode *modep, struct audit_list *audit_list)
 	  GLRO(dl_debug_mask) = 0;
 	}
 
-      if (mode != normal)
+      if (state->mode != normal)
 	_exit (5);
     }
   /* If we have to run the dynamic linker in debugging mode and the
      LD_DEBUG_OUTPUT environment variable is given, we write the debug
      messages to this file.  */
-  else if (any_debug && debug_output != NULL)
+  else if (state->any_debug && debug_output != NULL)
     {
       const int flags = O_WRONLY | O_APPEND | O_CREAT | O_NOFOLLOW;
       size_t name_len = strlen (debug_output);
-- 
2.25.4

[PATCH 07/30] elf: Move ld.so error/help output to _dl_usage

[Florian Weimer]

Also add a comment to elf/Makefile, explaining why we cannot use
config.status for autoconf template processing.
---
 elf/Makefile   |  9 ++++++++-
 elf/dl-main.h  |  5 +++++
 elf/dl-usage.c | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 elf/rtld.c     | 25 +------------------------
 4 files changed, 64 insertions(+), 25 deletions(-)
 create mode 100644 elf/dl-usage.c

diff --git a/elf/Makefile b/elf/Makefile
index b56fe0c0b7..85f7e08e00 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -59,7 +59,7 @@ elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \
 # ld.so uses those routines, plus some special stuff for being the program
 # interpreter and operating independent of libc.
 rtld-routines	= rtld $(all-dl-routines) dl-sysdep dl-environ dl-minimal \
-  dl-error-minimal dl-conflict dl-hwcaps
+  dl-error-minimal dl-conflict dl-hwcaps dl-usage
 all-rtld-routines = $(rtld-routines) $(sysdep-rtld-routines)
 
 CFLAGS-dl-runtime.c += -fexceptions -fasynchronous-unwind-tables
@@ -605,6 +605,12 @@ ldso_install: $(inst_rtlddir)/$(rtld-installed-name)
 endif
 
 
+# Workarounds for ${exec_prefix} expansion in configure variables.
+# config.status cannot be used directly for processing ldd.bash.in or
+# expanding variables such as sysconfdir because the expansion
+# contains the literal string ${exec_prefix}, which is not valid in C
+# headers or installed shell scripts.
+
 ldd-rewrite = -e 's%@RTLD@%$(rtlddir)/$(rtld-installed-name)%g' \
 	      -e 's%@VERSION@%$(version)%g' \
 	      -e 's|@PKGVERSION@|$(PKGVERSION)|g' \
@@ -642,6 +648,7 @@ libof-ldconfig = ldconfig
 CFLAGS-dl-cache.c += $(SYSCONF-FLAGS)
 CFLAGS-cache.c += $(SYSCONF-FLAGS)
 CFLAGS-rtld.c += $(SYSCONF-FLAGS)
+CFLAGS-dl-usage.c += $(SYSCONF-FLAGS)
 
 cpp-srcs-left := $(all-rtld-routines:=.os)
 lib := rtld
diff --git a/elf/dl-main.h b/elf/dl-main.h
index ad9250171f..681f366871 100644
--- a/elf/dl-main.h
+++ b/elf/dl-main.h
@@ -19,7 +19,9 @@
 #ifndef _DL_MAIN
 #define _DL_MAIN
 
+#include <ldsodefs.h>
 #include <limits.h>
+#include <stdlib.h>
 
 /* Length limits for names and paths, to protect the dynamic linker,
    particularly when __libc_enable_secure is active.  */
@@ -92,4 +94,7 @@ call_init_paths (const struct dl_main_state *state)
   _dl_init_paths (state->library_path);
 }
 
+/* Print ld.so usage information and exit.  */
+void _dl_usage (void) attribute_hidden __attribute__ ((__noreturn__));
+
 #endif /* _DL_MAIN */
diff --git a/elf/dl-usage.c b/elf/dl-usage.c
new file mode 100644
index 0000000000..e03f183622
--- /dev/null
+++ b/elf/dl-usage.c
@@ -0,0 +1,50 @@
+/* Print usage information and help for ld.so.
+   Copyright (C) 1995-2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-cache.h>
+#include <dl-main.h>
+#include <ldsodefs.h>
+
+void
+_dl_usage (void)
+{
+  _dl_fatal_printf ("\
+Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\n\
+You have invoked `ld.so', the helper program for shared library executables.\n\
+This program usually lives in the file `/lib/ld.so', and special directives\n\
+in executable files using ELF shared libraries tell the system's program\n\
+loader to load the helper program from this file.  This helper program loads\n\
+the shared libraries needed by the program executable, prepares the program\n\
+to run, and runs it.  You may invoke this helper program directly from the\n\
+command line to load and run an ELF executable file; this is like executing\n\
+that file itself, but always uses this helper program from the file you\n\
+specified, instead of the helper program file specified in the executable\n\
+file you run.  This is mostly of use for maintainers to test new versions\n\
+of this helper program; chances are you did not intend to run this program.\n\
+\n\
+  --list                list all dependencies and how they are resolved\n\
+  --verify              verify that given object really is a dynamically linked\n\
+                        object we can handle\n\
+  --inhibit-cache       Do not use " LD_SO_CACHE "\n\
+  --library-path PATH   use given PATH instead of content of the environment\n\
+                        variable LD_LIBRARY_PATH\n\
+  --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\
+                        in LIST\n\
+  --audit LIST          use objects named in LIST as auditors\n\
+  --preload LIST        preload objects named in LIST\n");
+}
diff --git a/elf/rtld.c b/elf/rtld.c
index e3556eabc4..5fe6ca969a 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -1216,30 +1216,7 @@ dl_main (const ElfW(Phdr) *phdr,
       /* If we have no further argument the program was called incorrectly.
 	 Grant the user some education.  */
       if (_dl_argc < 2)
-	_dl_fatal_printf ("\
-Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\n\
-You have invoked `ld.so', the helper program for shared library executables.\n\
-This program usually lives in the file `/lib/ld.so', and special directives\n\
-in executable files using ELF shared libraries tell the system's program\n\
-loader to load the helper program from this file.  This helper program loads\n\
-the shared libraries needed by the program executable, prepares the program\n\
-to run, and runs it.  You may invoke this helper program directly from the\n\
-command line to load and run an ELF executable file; this is like executing\n\
-that file itself, but always uses this helper program from the file you\n\
-specified, instead of the helper program file specified in the executable\n\
-file you run.  This is mostly of use for maintainers to test new versions\n\
-of this helper program; chances are you did not intend to run this program.\n\
-\n\
-  --list                list all dependencies and how they are resolved\n\
-  --verify              verify that given object really is a dynamically linked\n\
-			object we can handle\n\
-  --inhibit-cache       Do not use " LD_SO_CACHE "\n\
-  --library-path PATH   use given PATH instead of content of the environment\n\
-			variable LD_LIBRARY_PATH\n\
-  --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\
-			in LIST\n\
-  --audit LIST          use objects named in LIST as auditors\n\
-  --preload LIST        preload objects named in LIST\n");
+	_dl_usage ();
 
       ++_dl_skip_args;
       --_dl_argc;
-- 
2.25.4

[PATCH 08/30] elf: Record whether paths come from LD_LIBRARY_PATH or --library-path

[Florian Weimer]

This allows more precise diagnostics.
---
 elf/dl-load.c              | 4 ++--
 elf/dl-main.h              | 5 ++++-
 elf/dl-support.c           | 2 +-
 elf/rtld.c                 | 3 +++
 sysdeps/generic/ldsodefs.h | 6 ++++--
 5 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/elf/dl-load.c b/elf/dl-load.c
index 34d3b02a95..2339ada485 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -682,7 +682,7 @@ cache_rpath (struct link_map *l,
 
 
 void
-_dl_init_paths (const char *llp)
+_dl_init_paths (const char *llp, const char *source)
 {
   size_t idx;
   const char *strp;
@@ -820,7 +820,7 @@ _dl_init_paths (const char *llp)
 	}
 
       (void) fillin_rpath (llp_tmp, env_path_list.dirs, ":;",
-			   "LD_LIBRARY_PATH", NULL, l);
+			   source, NULL, l);
 
       if (env_path_list.dirs[0] == NULL)
 	{
diff --git a/elf/dl-main.h b/elf/dl-main.h
index 681f366871..68dd27d0d7 100644
--- a/elf/dl-main.h
+++ b/elf/dl-main.h
@@ -71,6 +71,9 @@ struct dl_main_state
   /* The library search path.  */
   const char *library_path;
 
+  /* Where library_path comes from.  LD_LIBRARY_PATH or --library.  */
+  const char *library_path_source;
+
   /* The list preloaded objects from LD_PRELOAD.  */
   const char *preloadlist;
 
@@ -91,7 +94,7 @@ struct dl_main_state
 static inline void
 call_init_paths (const struct dl_main_state *state)
 {
-  _dl_init_paths (state->library_path);
+  _dl_init_paths (state->library_path, state->library_path_source);
 }
 
 /* Print ld.so usage information and exit.  */
diff --git a/elf/dl-support.c b/elf/dl-support.c
index 7704c101c5..afbc94df54 100644
--- a/elf/dl-support.c
+++ b/elf/dl-support.c
@@ -323,7 +323,7 @@ _dl_non_dynamic_init (void)
 
   /* Initialize the data structures for the search paths for shared
      objects.  */
-  _dl_init_paths (getenv ("LD_LIBRARY_PATH"));
+  _dl_init_paths (getenv ("LD_LIBRARY_PATH"), "LD_LIBRARY_PATH");
 
   /* Remember the last search directory added at startup.  */
   _dl_init_all_dirs = GL(dl_all_dirs);
diff --git a/elf/rtld.c b/elf/rtld.c
index 5fe6ca969a..eac9a1e743 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -269,6 +269,7 @@ dl_main_state_init (struct dl_main_state *state)
 {
   audit_list_init (&state->audit_list);
   state->library_path = NULL;
+  state->library_path_source = NULL;
   state->preloadlist = NULL;
   state->preloadarg = NULL;
   state->mode = normal;
@@ -1181,6 +1182,7 @@ dl_main (const ElfW(Phdr) *phdr,
 		 && _dl_argc > 2)
 	  {
 	    state.library_path = _dl_argv[2];
+	    state.library_path_source = "--library_path";
 
 	    _dl_skip_args += 2;
 	    _dl_argc -= 2;
@@ -2609,6 +2611,7 @@ process_envvars (struct dl_main_state *state)
 	      && memcmp (envline, "LIBRARY_PATH", 12) == 0)
 	    {
 	      state->library_path = &envline[13];
+	      state->library_path_source = "LD_LIBRARY_PATH";
 	      break;
 	    }
 
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 42fe912f47..0f23352302 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -1034,8 +1034,10 @@ rtld_hidden_proto (_dl_debug_state)
 extern struct r_debug *_dl_debug_initialize (ElfW(Addr) ldbase, Lmid_t ns)
      attribute_hidden;
 
-/* Initialize the basic data structure for the search paths.  */
-extern void _dl_init_paths (const char *library_path) attribute_hidden;
+/* Initialize the basic data structure for the search paths.  SOURCE
+   is either "LD_LIBRARY_PATH" or "--library-path".  */
+extern void _dl_init_paths (const char *library_path, const char *source)
+  attribute_hidden;
 
 /* Gather the information needed to install the profiling tables and start
    the timers.  */
-- 
2.25.4

[PATCH 09/30] elf: Implement ld.so --help

[Florian Weimer]

--help processing is deferred to the point where the executable has
been loaded, so that it is possible to eventually include information
from the main executable in the help output.

As suggested in the GNU command-line interface guidelines, the help
message is printed to standard output, and the exit status is
successful.

Handle usage errors closer to the GNU command-line interface
guidelines.
---
 elf/dl-main.h  |  9 +++++++--
 elf/dl-usage.c | 24 ++++++++++++++++++----
 elf/rtld.c     | 55 ++++++++++++++++++++++++++++++++++++++++++--------
 3 files changed, 74 insertions(+), 14 deletions(-)

diff --git a/elf/dl-main.h b/elf/dl-main.h
index 68dd27d0d7..71ca5114de 100644
--- a/elf/dl-main.h
+++ b/elf/dl-main.h
@@ -60,7 +60,7 @@ struct audit_list
 };
 
 /* This is a list of all the modes the dynamic loader can be in.  */
-enum mode { normal, list, verify, trace };
+enum mode { normal, list, verify, trace, rtld_help };
 
 /* Aggregated state information extracted from environment variables
    and the ld.so command line.  */
@@ -98,6 +98,11 @@ call_init_paths (const struct dl_main_state *state)
 }
 
 /* Print ld.so usage information and exit.  */
-void _dl_usage (void) attribute_hidden __attribute__ ((__noreturn__));
+void _dl_usage (const char *argv0, const char *wrong_option)
+  attribute_hidden __attribute__ ((__noreturn__));
+
+/* Print ld.so --help output and exit.  */
+void _dl_help (const char *argv0, struct dl_main_state *state)
+  attribute_hidden __attribute__ ((__noreturn__));
 
 #endif /* _DL_MAIN */
diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index e03f183622..e1dc5d33b2 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -19,12 +19,24 @@
 #include <dl-cache.h>
 #include <dl-main.h>
 #include <ldsodefs.h>
+#include <unistd.h>
 
 void
-_dl_usage (void)
+_dl_usage (const char *argv0, const char *wrong_option)
 {
-  _dl_fatal_printf ("\
-Usage: ld.so [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\n\
+  if (wrong_option != NULL)
+    _dl_error_printf ("%s: unrecognized option '%s'\n", argv0, wrong_option);
+  else
+    _dl_error_printf ("%s: missing program name\n", argv0);
+  _dl_error_printf ("Try '%s --help' for more information.\n", argv0);
+  _exit (1);
+}
+
+void
+_dl_help (const char *argv0, struct dl_main_state *state)
+{
+  _dl_printf ("\
+Usage: %s [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\n\
 You have invoked `ld.so', the helper program for shared library executables.\n\
 This program usually lives in the file `/lib/ld.so', and special directives\n\
 in executable files using ELF shared libraries tell the system's program\n\
@@ -46,5 +58,9 @@ of this helper program; chances are you did not intend to run this program.\n\
   --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\
                         in LIST\n\
   --audit LIST          use objects named in LIST as auditors\n\
-  --preload LIST        preload objects named in LIST\n");
+  --preload LIST        preload objects named in LIST\n\
+  --help                display this help and exit\n\
+",
+              argv0);
+  _exit (0);
 }
diff --git a/elf/rtld.c b/elf/rtld.c
index eac9a1e743..610203d5d2 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -1131,6 +1131,7 @@ dl_main (const ElfW(Phdr) *phdr,
   _dl_starting_up = 1;
 #endif
 
+  const char *argv0 = _dl_argv[0];
   if (*user_entry == (ElfW(Addr)) ENTRY_POINT)
     {
       /* Ho ho.  We are not the program interpreter!  We are the program
@@ -1156,8 +1157,12 @@ dl_main (const ElfW(Phdr) *phdr,
       while (_dl_argc > 1)
 	if (! strcmp (_dl_argv[1], "--list"))
 	  {
-	    state.mode = list;
-	    GLRO(dl_lazy) = -1;	/* This means do no dependency analysis.  */
+	    if (state.mode != rtld_help)
+	      {
+	       state.mode = list;
+		/* This means do no dependency analysis.  */
+		GLRO(dl_lazy) = -1;
+	      }
 
 	    ++_dl_skip_args;
 	    --_dl_argc;
@@ -1165,7 +1170,8 @@ dl_main (const ElfW(Phdr) *phdr,
 	  }
 	else if (! strcmp (_dl_argv[1], "--verify"))
 	  {
-	    state.mode = verify;
+	    if (state.mode != rtld_help)
+	      state.mode = verify;
 
 	    ++_dl_skip_args;
 	    --_dl_argc;
@@ -1212,13 +1218,34 @@ dl_main (const ElfW(Phdr) *phdr,
 	    _dl_argc -= 2;
 	    _dl_argv += 2;
 	  }
+	else if (strcmp (_dl_argv[1], "--help") == 0)
+	  {
+	    state.mode = rtld_help;
+	    --_dl_argc;
+	    ++_dl_argv;
+	  }
+	else if (_dl_argv[1][0] == '-' && _dl_argv[1][1] == '-')
+	  {
+	   if (_dl_argv[1][1] == '\0')
+	     /* End of option list.  */
+	     break;
+	   else
+	     /* Unrecognized option.  */
+	     _dl_usage (argv0, _dl_argv[1]);
+	  }
 	else
 	  break;
 
       /* If we have no further argument the program was called incorrectly.
 	 Grant the user some education.  */
       if (_dl_argc < 2)
-	_dl_usage ();
+	{
+	  if (state.mode == rtld_help)
+	    /* --help without an executable is not an error.  */
+	    _dl_help (argv0, &state);
+	  else
+	    _dl_usage (argv0, NULL);
+	}
 
       ++_dl_skip_args;
       --_dl_argc;
@@ -1243,7 +1270,7 @@ dl_main (const ElfW(Phdr) *phdr,
 	    break;
 	  }
 
-      if (__builtin_expect (state.mode, normal) == verify)
+      if (state.mode == verify || state.mode == rtld_help)
 	{
 	  const char *objname;
 	  const char *err_str = NULL;
@@ -1256,9 +1283,16 @@ dl_main (const ElfW(Phdr) *phdr,
 	  (void) _dl_catch_error (&objname, &err_str, &malloced, map_doit,
 				  &args);
 	  if (__glibc_unlikely (err_str != NULL))
-	    /* We don't free the returned string, the programs stops
-	       anyway.  */
-	    _exit (EXIT_FAILURE);
+	    {
+	      /* We don't free the returned string, the programs stops
+		 anyway.  */
+	      if (state.mode == rtld_help)
+		/* Mask the failure to load the main object.  The help
+		   message contains less information in this case.  */
+		_dl_help (argv0, &state);
+	      else
+		_exit (EXIT_FAILURE);
+	    }
 	}
       else
 	{
@@ -1607,6 +1641,11 @@ ERROR: '%s': cannot process note segment.\n", _dl_argv[0]);
   audit_list_add_dynamic_tag (&state.audit_list, main_map, DT_AUDIT);
   audit_list_add_dynamic_tag (&state.audit_list, main_map, DT_DEPAUDIT);
 
+  /* At this point, all data has been obtained that is included in the
+     --help output.  */
+  if (__builtin_expect (state.mode, normal) == rtld_help)
+    _dl_help (argv0, &state);
+
   /* If we have auditing DSOs to load, do it now.  */
   bool need_security_init = true;
   if (state.audit_list.length > 0)
-- 
2.25.4

[PATCH 10/30] elf: Implement ld.so --version

[Florian Weimer]

This prints out version information for the dynamic loader and
exits immediately, without further command line processing
(which seems to match what some GNU tools do).
---
 elf/dl-main.h  |  3 +++
 elf/dl-usage.c | 15 +++++++++++++++
 elf/rtld.c     |  2 ++
 3 files changed, 20 insertions(+)

diff --git a/elf/dl-main.h b/elf/dl-main.h
index 71ca5114de..0df849d3cd 100644
--- a/elf/dl-main.h
+++ b/elf/dl-main.h
@@ -101,6 +101,9 @@ call_init_paths (const struct dl_main_state *state)
 void _dl_usage (const char *argv0, const char *wrong_option)
   attribute_hidden __attribute__ ((__noreturn__));
 
+/* Print ld.so version information and exit.  */
+void _dl_version (void) attribute_hidden __attribute__ ((__noreturn__));
+
 /* Print ld.so --help output and exit.  */
 void _dl_help (const char *argv0, struct dl_main_state *state)
   attribute_hidden __attribute__ ((__noreturn__));
diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index e1dc5d33b2..c8d182c442 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -20,6 +20,7 @@
 #include <dl-main.h>
 #include <ldsodefs.h>
 #include <unistd.h>
+#include "version.h"
 
 void
 _dl_usage (const char *argv0, const char *wrong_option)
@@ -32,6 +33,19 @@ _dl_usage (const char *argv0, const char *wrong_option)
   _exit (1);
 }
 
+void
+_dl_version (void)
+{
+  _dl_printf ("\
+ld.so " PKGVERSION RELEASE " release version " VERSION ".\n\
+Copyright (C) 2020 Free Software Foundation, Inc.\n\
+This is free software; see the source for copying conditions.\n\
+There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A\n\
+PARTICULAR PURPOSE.\n\
+");
+  _exit (0);
+}
+
 void
 _dl_help (const char *argv0, struct dl_main_state *state)
 {
@@ -60,6 +74,7 @@ of this helper program; chances are you did not intend to run this program.\n\
   --audit LIST          use objects named in LIST as auditors\n\
   --preload LIST        preload objects named in LIST\n\
   --help                display this help and exit\n\
+  --version             output version information and exit\n\
 ",
               argv0);
   _exit (0);
diff --git a/elf/rtld.c b/elf/rtld.c
index 610203d5d2..06bc8eca9a 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -1224,6 +1224,8 @@ dl_main (const ElfW(Phdr) *phdr,
 	    --_dl_argc;
 	    ++_dl_argv;
 	  }
+	else if (strcmp (_dl_argv[1], "--version") == 0)
+	  _dl_version ();
 	else if (_dl_argv[1][0] == '-' && _dl_argv[1][1] == '-')
 	  {
 	   if (_dl_argv[1][1] == '\0')
-- 
2.25.4

[PATCH 11/30] scripts/update-copyrights: Update csu/version.c, elf/dl-usage.c

[Florian Weimer]

---
 scripts/update-copyrights | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/scripts/update-copyrights b/scripts/update-copyrights
index 5ab9489511..7cca0f2c3d 100755
--- a/scripts/update-copyrights
+++ b/scripts/update-copyrights
@@ -70,6 +70,12 @@ for f in $files; do
       # Pre-1991 gaps in copyright years, so cannot use a single range.
       UPDATE_COPYRIGHT_USE_INTERVALS=1 "$update_script" "$f"
       ;;
+    csu/version.c | elf/dl-usage.c)
+      # Update the copyright string in the output message.
+      year="$(date +%Y)"
+      sed -i 's/^Copyright (C) [0-9]\{4\} /Copyright (C) '"$year"' /' $f
+      "$update_script" "$f"
+      ;;
     *)
       "$update_script" "$f"
       ;;
-- 
2.25.4

[PATCH 12/30] elf: Use the term "program interpreter" in the ld.so help message

[Florian Weimer]

This is the term that the ELF standard itself uses.
---
 elf/dl-usage.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index c8d182c442..0a12c631a6 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -51,17 +51,17 @@ _dl_help (const char *argv0, struct dl_main_state *state)
 {
   _dl_printf ("\
 Usage: %s [OPTION]... EXECUTABLE-FILE [ARGS-FOR-PROGRAM...]\n\
-You have invoked `ld.so', the helper program for shared library executables.\n\
-This program usually lives in the file `/lib/ld.so', and special directives\n\
-in executable files using ELF shared libraries tell the system's program\n\
-loader to load the helper program from this file.  This helper program loads\n\
-the shared libraries needed by the program executable, prepares the program\n\
-to run, and runs it.  You may invoke this helper program directly from the\n\
+You have invoked 'ld.so', the program interpreter for dynamically-linked\n\
+ELF programs.  Usually, the program interpreter is invoked automatically\n\
+when a dynamically-linked executable is started.\n\
+\n\
+You may invoke the program interpreter program directly from the\n\
 command line to load and run an ELF executable file; this is like executing\n\
-that file itself, but always uses this helper program from the file you\n\
-specified, instead of the helper program file specified in the executable\n\
-file you run.  This is mostly of use for maintainers to test new versions\n\
-of this helper program; chances are you did not intend to run this program.\n\
+that file itself, but always uses the invoked program interpreter you\n\
+invoked, instead of the program interpreter specified in the executable\n\
+file you run.  Invoking the program interpreter directly provides access to\n\
+additional diagnostics, and changing the dynamic linker behavior without\n\
+setting environment variables (which would be inherted by subprocesses).\n\
 \n\
   --list                list all dependencies and how they are resolved\n\
   --verify              verify that given object really is a dynamically linked\n\
-- 
2.25.4

[PATCH 13/30] elf: Print the full name of the dynamic loader in the ld.so help message

[Florian Weimer]

This requires defining a macro for the full path, matching the
-Wl,--dynamic-link= arguments used for linking glibc programs,
and ldd script.
---
 elf/Makefile   | 3 ++-
 elf/dl-usage.c | 2 ++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/elf/Makefile b/elf/Makefile
index 85f7e08e00..d2f58c8ce6 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -648,7 +648,8 @@ libof-ldconfig = ldconfig
 CFLAGS-dl-cache.c += $(SYSCONF-FLAGS)
 CFLAGS-cache.c += $(SYSCONF-FLAGS)
 CFLAGS-rtld.c += $(SYSCONF-FLAGS)
-CFLAGS-dl-usage.c += $(SYSCONF-FLAGS)
+CFLAGS-dl-usage.c += $(SYSCONF-FLAGS) \
+  -D'RTLD="$(rtlddir)/$(rtld-installed-name)"'
 
 cpp-srcs-left := $(all-rtld-routines:=.os)
 lib := rtld
diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index 0a12c631a6..5caf9794c6 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -75,6 +75,8 @@ setting environment variables (which would be inherted by subprocesses).\n\
   --preload LIST        preload objects named in LIST\n\
   --help                display this help and exit\n\
   --version             output version information and exit\n\
+\n\
+This program interpreter self-identifies as: " RTLD "\n\
 ",
               argv0);
   _exit (0);
-- 
2.25.4

[PATCH 14/30] elf: Make __rtld_env_path_list and __rtld_search_dirs global variables

[Florian Weimer]

They have been renamed from env_path_list and rtld_search_dirs to
avoid linknamespace issues.

This change will allow future use these variables in diagnostics.
---
 elf/dl-load.c  | 53 +++++++++++++++++++++++++-------------------------
 include/link.h |  4 ++++
 2 files changed, 31 insertions(+), 26 deletions(-)

diff --git a/elf/dl-load.c b/elf/dl-load.c
index 2339ada485..675d5bd48b 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -98,7 +98,7 @@ int __stack_prot attribute_hidden attribute_relro
 
 
 /* This is the decomposed LD_LIBRARY_PATH search path.  */
-static struct r_search_path_struct env_path_list attribute_relro;
+struct r_search_path_struct __rtld_env_path_list attribute_relro;
 
 /* List of the hardware capabilities we might end up using.  */
 #ifdef SHARED
@@ -442,7 +442,7 @@ add_name_to_object (struct link_map *l, const char *name)
 }
 
 /* Standard search directories.  */
-static struct r_search_path_struct rtld_search_dirs attribute_relro;
+struct r_search_path_struct __rtld_search_dirs attribute_relro;
 
 static size_t max_dirnamelen;
 
@@ -702,9 +702,9 @@ _dl_init_paths (const char *llp, const char *source)
 #endif
 
   /* First set up the rest of the default search directory entries.  */
-  aelem = rtld_search_dirs.dirs = (struct r_search_path_elem **)
+  aelem = __rtld_search_dirs.dirs = (struct r_search_path_elem **)
     malloc ((nsystem_dirs_len + 1) * sizeof (struct r_search_path_elem *));
-  if (rtld_search_dirs.dirs == NULL)
+  if (__rtld_search_dirs.dirs == NULL)
     {
       errstring = N_("cannot create search path array");
     signal_error:
@@ -715,16 +715,17 @@ _dl_init_paths (const char *llp, const char *source)
 		 + ncapstr * sizeof (enum r_dir_status))
 		/ sizeof (struct r_search_path_elem));
 
-  rtld_search_dirs.dirs[0] = malloc (nsystem_dirs_len * round_size
-				     * sizeof (*rtld_search_dirs.dirs[0]));
-  if (rtld_search_dirs.dirs[0] == NULL)
+  __rtld_search_dirs.dirs[0]
+    = malloc (nsystem_dirs_len * round_size
+	      * sizeof (*__rtld_search_dirs.dirs[0]));
+  if (__rtld_search_dirs.dirs[0] == NULL)
     {
       errstring = N_("cannot create cache for search path");
       goto signal_error;
     }
 
-  rtld_search_dirs.malloced = 0;
-  pelem = GL(dl_all_dirs) = rtld_search_dirs.dirs[0];
+  __rtld_search_dirs.malloced = 0;
+  pelem = GL(dl_all_dirs) = __rtld_search_dirs.dirs[0];
   strp = system_dirs;
   idx = 0;
 
@@ -811,27 +812,27 @@ _dl_init_paths (const char *llp, const char *source)
 	if (*cp == ':' || *cp == ';')
 	  ++nllp;
 
-      env_path_list.dirs = (struct r_search_path_elem **)
+      __rtld_env_path_list.dirs = (struct r_search_path_elem **)
 	malloc ((nllp + 1) * sizeof (struct r_search_path_elem *));
-      if (env_path_list.dirs == NULL)
+      if (__rtld_env_path_list.dirs == NULL)
 	{
 	  errstring = N_("cannot create cache for search path");
 	  goto signal_error;
 	}
 
-      (void) fillin_rpath (llp_tmp, env_path_list.dirs, ":;",
+      (void) fillin_rpath (llp_tmp, __rtld_env_path_list.dirs, ":;",
 			   source, NULL, l);
 
-      if (env_path_list.dirs[0] == NULL)
+      if (__rtld_env_path_list.dirs[0] == NULL)
 	{
-	  free (env_path_list.dirs);
-	  env_path_list.dirs = (void *) -1;
+	  free (__rtld_env_path_list.dirs);
+	  __rtld_env_path_list.dirs = (void *) -1;
 	}
 
-      env_path_list.malloced = 0;
+      __rtld_env_path_list.malloced = 0;
     }
   else
-    env_path_list.dirs = (void *) -1;
+    __rtld_env_path_list.dirs = (void *) -1;
 }
 
 
@@ -1918,9 +1919,9 @@ open_path (const char *name, size_t namelen, int mode,
       if (sps->malloced)
 	free (sps->dirs);
 
-      /* rtld_search_dirs and env_path_list are attribute_relro, therefore
-	 avoid writing into it.  */
-      if (sps != &rtld_search_dirs && sps != &env_path_list)
+      /* __rtld_search_dirs and __rtld_env_path_list are
+	 attribute_relro, therefore avoid writing to them.  */
+      if (sps != &__rtld_search_dirs && sps != &__rtld_env_path_list)
 	sps->dirs = (void *) -1;
     }
 
@@ -2068,8 +2069,8 @@ _dl_map_object (struct link_map *loader, const char *name,
 	}
 
       /* Try the LD_LIBRARY_PATH environment variable.  */
-      if (fd == -1 && env_path_list.dirs != (void *) -1)
-	fd = open_path (name, namelen, mode, &env_path_list,
+      if (fd == -1 && __rtld_env_path_list.dirs != (void *) -1)
+	fd = open_path (name, namelen, mode, &__rtld_env_path_list,
 			&realname, &fb,
 			loader ?: GL(dl_ns)[LM_ID_BASE]._ns_loaded,
 			LA_SER_LIBPATH, &found_other_class);
@@ -2158,8 +2159,8 @@ _dl_map_object (struct link_map *loader, const char *name,
       if (fd == -1
 	  && ((l = loader ?: GL(dl_ns)[nsid]._ns_loaded) == NULL
 	      || __glibc_likely (!(l->l_flags_1 & DF_1_NODEFLIB)))
-	  && rtld_search_dirs.dirs != (void *) -1)
-	fd = open_path (name, namelen, mode, &rtld_search_dirs,
+	  && __rtld_search_dirs.dirs != (void *) -1)
+	fd = open_path (name, namelen, mode, &__rtld_search_dirs,
 			&realname, &fb, l, LA_SER_DEFAULT, &found_other_class);
 
       /* Add another newline when we are tracing the library loading.  */
@@ -2327,7 +2328,7 @@ _dl_rtld_di_serinfo (struct link_map *loader, Dl_serinfo *si, bool counting)
     }
 
   /* Try the LD_LIBRARY_PATH environment variable.  */
-  add_path (&p, &env_path_list, XXX_ENV);
+  add_path (&p, &__rtld_env_path_list, XXX_ENV);
 
   /* Look at the RUNPATH information for this binary.  */
   if (cache_rpath (loader, &loader->l_runpath_dirs, DT_RUNPATH, "RUNPATH"))
@@ -2339,7 +2340,7 @@ _dl_rtld_di_serinfo (struct link_map *loader, Dl_serinfo *si, bool counting)
 
   /* Finally, try the default path.  */
   if (!(loader->l_flags_1 & DF_1_NODEFLIB))
-    add_path (&p, &rtld_search_dirs, XXX_default);
+    add_path (&p, &__rtld_search_dirs, XXX_default);
 
   if (counting)
     /* Count the struct size before the string area, which we didn't
diff --git a/include/link.h b/include/link.h
index aea268439c..d4714bc28d 100644
--- a/include/link.h
+++ b/include/link.h
@@ -79,6 +79,10 @@ struct r_search_path_struct
     int malloced;
   };
 
+/* Search path information computed by _dl_init_paths.  */
+extern struct r_search_path_struct __rtld_search_dirs attribute_hidden;
+extern struct r_search_path_struct __rtld_env_path_list attribute_hidden;
+
 /* Structure describing a loaded shared object.  The `l_next' and `l_prev'
    members form a chain of all the shared objects loaded at startup.
 
-- 
2.25.4

[PATCH 15/30] elf: Add library search path information to ld.so --help

[Florian Weimer]

---
 elf/dl-usage.c | 56 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 56 insertions(+)

diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index 5caf9794c6..4712ab1c72 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -46,6 +46,61 @@ PARTICULAR PURPOSE.\n\
   _exit (0);
 }
 
+/* Print part of the library search path (from a single source).  */
+static void
+print_search_path_for_help_1 (struct r_search_path_elem **list)
+{
+  if (list == NULL || list == (void *) -1)
+    /* Path is missing or marked as inactive.  */
+    return;
+
+  for (; *list != NULL; ++list)
+    {
+      (void) _dl_write (STDOUT_FILENO, "  ", 2);
+      const char *name = (*list)->dirname;
+      size_t namelen = (*list)->dirnamelen;
+      if (namelen == 0)
+        {
+          /* The empty string denotes the current directory.  */
+          name = ".";
+          namelen = 1;
+        }
+      else if (namelen > 1)
+        /* Remove the trailing slash.  */
+        --namelen;
+      (void) _dl_write (STDOUT_FILENO, name, namelen);
+      _dl_printf (" (%s)\n", (*list)->what);
+    }
+}
+
+/* Prints the library search path.  See _dl_init_paths in dl-load.c
+   how this information is populated.  */
+static void
+print_search_path_for_help (struct dl_main_state *state)
+{
+  if (__rtld_search_dirs.dirs == NULL)
+    /* The run-time search paths have not yet been initialized.  */
+    _dl_init_paths (state->library_path, state->library_path_source);
+
+  _dl_printf ("\nShared library search path:\n");
+
+  /* The print order should reflect the processing in
+     _dl_map_object.  */
+
+  struct link_map *map = GL(dl_ns)[LM_ID_BASE]._ns_loaded;
+  if (map != NULL)
+    print_search_path_for_help_1 (map->l_rpath_dirs.dirs);
+
+  print_search_path_for_help_1 (__rtld_env_path_list.dirs);
+
+  if (map != NULL)
+    print_search_path_for_help_1 (map->l_runpath_dirs.dirs);
+
+  _dl_printf ("  (libraries located via %s)\n", LD_SO_CACHE);
+
+  print_search_path_for_help_1 (__rtld_search_dirs.dirs);
+}
+
 void
 _dl_help (const char *argv0, struct dl_main_state *state)
 {
@@ -79,5 +134,6 @@ setting environment variables (which would be inherted by subprocesses).\n\
 This program interpreter self-identifies as: " RTLD "\n\
 ",
               argv0);
+  print_search_path_for_help (state);
   _exit (0);
 }
-- 
2.25.4

[PATCH 16/30] elf: Enhance ld.so --help to print HWCAP subdirectories

[Florian Weimer]

---
 elf/dl-usage.c | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index 4712ab1c72..7ff642349c 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -22,6 +22,8 @@
 #include <unistd.h>
 #include "version.h"
 
+#include <dl-hwcaps.h>
+
 void
 _dl_usage (const char *argv0, const char *wrong_option)
 {
@@ -101,6 +103,65 @@ print_search_path_for_help (struct dl_main_state *state)
   print_search_path_for_help_1 (__rtld_search_dirs.dirs);
 }
 
+/* Helper function for printing flags associated with a HWCAP name.  */
+static void
+print_hwcap_1 (bool *first, bool active, const char *label)
+{
+  if (active)
+    {
+      if (*first)
+        {
+          _dl_printf (" (");
+          *first = false;
+        }
+      else
+        _dl_printf (", ");
+      _dl_printf ("%s", label);
+    }
+}
+
+/* Called after a series of print_hwcap_1 calls to emit the line
+   terminator.  */
+static void
+print_hwcap_1_finish (bool *first)
+{
+  if (*first)
+    _dl_printf ("\n");
+  else
+    _dl_printf (")\n");
+}
+
+/* Write a list of hwcap subdirectories to standard output.  See
+ _dl_important_hwcaps in dl-hwcaps.c.  */
+static void
+print_legacy_hwcap_directories (void)
+{
+  _dl_printf ("\n\
+Legacy HWCAP subdirectories under library search path directories:\n");
+
+  const char *platform = GLRO (dl_platform);
+  if (platform != NULL)
+    _dl_printf ("  %s (AT_PLATFORM)\n", platform);
+
+  _dl_printf ("  tls (supported, searched)\n");
+
+  uint64_t hwcap_mask = GET_HWCAP_MASK();
+  uint64_t searched = GLRO (dl_hwcap) & hwcap_mask;
+  for (int n = 63; n >= 0; --n)
+    {
+      uint64_t bit = 1ULL << n;
+      if (HWCAP_IMPORTANT & bit)
+        {
+          _dl_printf ("  %s", _dl_hwcap_string (n));
+          bool first = true;
+          print_hwcap_1 (&first, GLRO (dl_hwcap) & bit, "supported");
+          print_hwcap_1 (&first, !(hwcap_mask & bit), "masked");
+          print_hwcap_1 (&first, searched & bit, "searched");
+          print_hwcap_1_finish (&first);
+        }
+    }
+}
+
 void
 _dl_help (const char *argv0, struct dl_main_state *state)
 {
@@ -135,5 +196,6 @@ This program interpreter self-identifies as: " RTLD "\n\
 ",
               argv0);
   print_search_path_for_help (state);
+  print_legacy_hwcap_directories ();
   _exit (0);
 }
-- 
2.25.4

[PATCH 17/30] elf: Do not pass GLRO(dl_platform), GLRO(dl_platformlen) to _dl_important_hwcaps

[Florian Weimer]

In the current code, the function can easily obtain the information
on its own.
---
 elf/dl-hwcaps.c            | 11 +++++------
 elf/dl-load.c              |  3 +--
 sysdeps/generic/ldsodefs.h | 12 ++++++------
 3 files changed, 12 insertions(+), 14 deletions(-)

diff --git a/elf/dl-hwcaps.c b/elf/dl-hwcaps.c
index 6df9efb255..44dbac099f 100644
--- a/elf/dl-hwcaps.c
+++ b/elf/dl-hwcaps.c
@@ -28,13 +28,12 @@
 
 /* Return an array of useful/necessary hardware capability names.  */
 const struct r_strlenpair *
-_dl_important_hwcaps (const char *platform, size_t platform_len, size_t *sz,
-		      size_t *max_capstrlen)
+_dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
 {
   uint64_t hwcap_mask = GET_HWCAP_MASK();
   /* Determine how many important bits are set.  */
   uint64_t masked = GLRO(dl_hwcap) & hwcap_mask;
-  size_t cnt = platform != NULL;
+  size_t cnt = GLRO (dl_platform) != NULL;
   size_t n, m;
   size_t total;
   struct r_strlenpair *result;
@@ -60,10 +59,10 @@ _dl_important_hwcaps (const char *platform, size_t platform_len, size_t *sz,
 	masked ^= 1ULL << n;
 	++m;
       }
-  if (platform != NULL)
+  if (GLRO (dl_platform) != NULL)
     {
-      temp[m].str = platform;
-      temp[m].len = platform_len;
+      temp[m].str = GLRO (dl_platform);
+      temp[m].len = GLRO (dl_platformlen);
       ++m;
     }
 
diff --git a/elf/dl-load.c b/elf/dl-load.c
index 675d5bd48b..6e5fa2af13 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -697,8 +697,7 @@ _dl_init_paths (const char *llp, const char *source)
 
 #ifdef SHARED
   /* Get the capabilities.  */
-  capstr = _dl_important_hwcaps (GLRO(dl_platform), GLRO(dl_platformlen),
-				 &ncapstr, &max_capstrlen);
+  capstr = _dl_important_hwcaps (&ncapstr, &max_capstrlen);
 #endif
 
   /* First set up the rest of the default search directory entries.  */
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 0f23352302..2de060848b 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -1059,12 +1059,12 @@ extern void _dl_show_auxv (void) attribute_hidden;
    other.  */
 extern char *_dl_next_ld_env_entry (char ***position) attribute_hidden;
 
-/* Return an array with the names of the important hardware capabilities.  */
-extern const struct r_strlenpair *_dl_important_hwcaps (const char *platform,
-							size_t paltform_len,
-							size_t *sz,
-							size_t *max_capstrlen)
-     attribute_hidden;
+/* Return an array with the names of the important hardware
+   capabilities.  The length of the array is written to *SZ, and the
+   maximum of all strings length is written to *MAX_CAPSTRLEN.  */
+const struct r_strlenpair *_dl_important_hwcaps (size_t *sz,
+						 size_t *max_capstrlen)
+  attribute_hidden;
 
 /* Look up NAME in ld.so.cache and return the file name stored there,
    or null if none is found.  Caller must free returned string.  */
-- 
2.25.4

[PATCH 18/30] elf: Add glibc-hwcaps support for LD_LIBRARY_PATH

[Florian Weimer]

This hacks non-power-set processing into _dl_important_hwcaps.
Once the legacy hwcaps handling goes away, the subdirectory
handling needs to be reworked, but it is premature to do this
while both approaches are still supported.
---
 elf/Makefile               |   5 +-
 elf/dl-hwcaps-subdirs.c    |  29 ++++++++
 elf/dl-hwcaps.c            | 138 +++++++++++++++++++++++++++++++-----
 elf/dl-hwcaps.h            |  83 ++++++++++++++++++++++
 elf/dl-hwcaps_split.c      |  77 ++++++++++++++++++++
 elf/dl-load.c              |   7 +-
 elf/dl-main.h              |  11 ++-
 elf/dl-support.c           |   5 +-
 elf/dl-usage.c             |  68 +++++++++++++++++-
 elf/rtld.c                 |  18 +++++
 elf/tst-dl-hwcaps_split.c  | 139 +++++++++++++++++++++++++++++++++++++
 sysdeps/generic/ldsodefs.h |  20 ++++--
 12 files changed, 570 insertions(+), 30 deletions(-)
 create mode 100644 elf/dl-hwcaps-subdirs.c
 create mode 100644 elf/dl-hwcaps_split.c
 create mode 100644 elf/tst-dl-hwcaps_split.c

diff --git a/elf/Makefile b/elf/Makefile
index d2f58c8ce6..728cb3b734 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -59,7 +59,8 @@ elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \
 # ld.so uses those routines, plus some special stuff for being the program
 # interpreter and operating independent of libc.
 rtld-routines	= rtld $(all-dl-routines) dl-sysdep dl-environ dl-minimal \
-  dl-error-minimal dl-conflict dl-hwcaps dl-usage
+  dl-error-minimal dl-conflict dl-hwcaps dl-hwcaps_split dl-hwcaps-subdirs \
+  dl-usage
 all-rtld-routines = $(rtld-routines) $(sysdep-rtld-routines)
 
 CFLAGS-dl-runtime.c += -fexceptions -fasynchronous-unwind-tables
@@ -210,7 +211,7 @@ tests-internal += loadtest unload unload2 circleload1 \
 	 neededtest neededtest2 neededtest3 neededtest4 \
 	 tst-tls3 tst-tls6 tst-tls7 tst-tls8 tst-dlmopen2 \
 	 tst-ptrguard1 tst-stackguard1 tst-libc_dlvsym \
-	 tst-create_format1
+	 tst-create_format1 tst-dl-hwcaps_split
 tests-container += tst-pldd tst-dlopen-tlsmodid-container \
   tst-dlopen-self-container
 test-srcs = tst-pathopt
diff --git a/elf/dl-hwcaps-subdirs.c b/elf/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..b142a3b826
--- /dev/null
+++ b/elf/dl-hwcaps-subdirs.c
@@ -0,0 +1,29 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  Generic version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+
+/* In the generic version, there are no subdirectories defined.  */
+
+const char _dl_hwcaps_subdirs[] = "";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  return 0;
+}
diff --git a/elf/dl-hwcaps.c b/elf/dl-hwcaps.c
index 44dbac099f..4de94759a2 100644
--- a/elf/dl-hwcaps.c
+++ b/elf/dl-hwcaps.c
@@ -26,20 +26,97 @@
 #include <dl-procinfo.h>
 #include <dl-hwcaps.h>
 
+/* This is the result of counting the substrings in a colon-separated
+   hwcaps string.  */
+struct count_hwcaps
+{
+  /* Number of substrings.  */
+  size_t count;
+
+  /* Sum of the individual substring lengths (without separates or
+     null terminators).  */
+  size_t total_length;
+
+  /* Maximum length of an individual substring.  */
+  size_t maximum_length;
+};
+
+/* Update *COUNTS according to the contents of HWCAPS.  Skip over
+   entries whose bit is not set in MASK.  */
+static void
+count_hwcaps (struct count_hwcaps *counts, const char *hwcaps,
+	      int32_t bitmask, const char *mask)
+{
+  struct dl_hwcaps_split_masked sp;
+  _dl_hwcaps_split_masked_init (&sp, hwcaps, bitmask, mask);
+  while (_dl_hwcaps_split_masked (&sp))
+    {
+      ++counts->count;
+      counts->total_length += sp.split.length;
+      if (sp.split.length > counts->maximum_length)
+	counts->maximum_length = sp.split.length;
+    }
+}
+
+/* State for copy_hwcaps.  Must be initialized to point to
+   the storage areas for the array and the strings themselves.  */
+struct copy_hwcaps
+{
+  struct r_strlenpair *next_pair;
+  char *next_string;
+};
+
+/* Copy HWCAPS into the string pairs and strings, advancing *TARGET.
+   Skip over entries whose bit is not set in MASK.  */
+static void
+copy_hwcaps (struct copy_hwcaps *target, const char *hwcaps,
+	     int32_t bitmask, const char *mask)
+{
+  struct dl_hwcaps_split_masked sp;
+  _dl_hwcaps_split_masked_init (&sp, hwcaps, bitmask, mask);
+  while (_dl_hwcaps_split_masked (&sp))
+    {
+      target->next_pair->str = target->next_string;
+      char *slash = __mempcpy (__mempcpy (target->next_string,
+					  GLIBC_HWCAPS_PREFIX,
+					  strlen (GLIBC_HWCAPS_PREFIX)),
+			       sp.split.segment, sp.split.length);
+      *slash = '/';
+      target->next_pair->len
+	= strlen (GLIBC_HWCAPS_PREFIX) + sp.split.length + 1;
+      ++target->next_pair;
+      target->next_string = slash + 1;
+    }
+}
+
 /* Return an array of useful/necessary hardware capability names.  */
 const struct r_strlenpair *
-_dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
+_dl_important_hwcaps (const char *glibc_hwcaps_prepend,
+		      const char *glibc_hwcaps_mask,
+		      size_t *sz, size_t *max_capstrlen)
 {
   uint64_t hwcap_mask = GET_HWCAP_MASK();
   /* Determine how many important bits are set.  */
   uint64_t masked = GLRO(dl_hwcap) & hwcap_mask;
   size_t cnt = GLRO (dl_platform) != NULL;
   size_t n, m;
-  size_t total;
   struct r_strlenpair *result;
   struct r_strlenpair *rp;
   char *cp;
 
+  /* glibc-hwcaps subdirectories.  These are exempted from the power
+     set construction below below.  */
+  int32_t hwcaps_subdirs_active = _dl_hwcaps_subdirs_active ();
+  struct count_hwcaps hwcaps_counts =  { 0, };
+  count_hwcaps (&hwcaps_counts, glibc_hwcaps_prepend, -1, NULL);
+  count_hwcaps (&hwcaps_counts, _dl_hwcaps_subdirs, hwcaps_subdirs_active,
+		glibc_hwcaps_mask);
+
+  /* Each hwcaps subdirectory has a GLIBC_HWCAPS_PREFIX string prefix
+     and a "/" suffix once stored in the result.  */
+  size_t total = (hwcaps_counts.count * (strlen (GLIBC_HWCAPS_PREFIX) + 1)
+		  + hwcaps_counts.total_length);
+
   /* Count the number of bits set in the masked value.  */
   for (n = 0; (~((1ULL << n) - 1) & masked) != 0; ++n)
     if ((masked & (1ULL << n)) != 0)
@@ -74,10 +151,10 @@ _dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
 
   /* Determine the total size of all strings together.  */
   if (cnt == 1)
-    total = temp[0].len + 1;
+    total += temp[0].len + 1;
   else
     {
-      total = temp[0].len + temp[cnt - 1].len + 2;
+      total += temp[0].len + temp[cnt - 1].len + 2;
       if (cnt > 2)
 	{
 	  total <<= 1;
@@ -94,26 +171,48 @@ _dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
 	}
     }
 
-  /* The result structure: we use a very compressed way to store the
-     various combinations of capability names.  */
-  *sz = 1 << cnt;
-  result = (struct r_strlenpair *) malloc (*sz * sizeof (*result) + total);
-  if (result == NULL)
+  *sz = hwcaps_counts.count + (1 << cnt);
+
+  /* This is the overall result, including both glibc-hwcaps
+     subdirectories and the legacy hwcaps subdirectories using the
+     power set construction.  */
+  struct r_strlenpair *overall_result
+    = malloc (*sz * sizeof (*result) + total);
+  if (overall_result == NULL)
     _dl_signal_error (ENOMEM, NULL, NULL,
 		      N_("cannot create capability list"));
 
+  /* Fill in the glibc-hwcaps subdirectories.  */
+  {
+    struct copy_hwcaps target;
+    target.next_pair = overall_result;
+    target.next_string = (char *) (overall_result + *sz);
+    copy_hwcaps (&target, glibc_hwcaps_prepend, -1, NULL);
+    copy_hwcaps (&target, _dl_hwcaps_subdirs,
+		 hwcaps_subdirs_active, glibc_hwcaps_mask);
+    /* Set up the write target for the power set construction.  */
+    result = target.next_pair;
+    cp = target.next_string;
+  }
+
+
+  /* Power set construction begins here.  We use a very compressed way
+     to store the various combinations of capability names.  */
+
   if (cnt == 1)
     {
-      result[0].str = (char *) (result + *sz);
+      result[0].str = cp;
       result[0].len = temp[0].len + 1;
-      result[1].str = (char *) (result + *sz);
+      result[1].str = cp;
       result[1].len = 0;
-      cp = __mempcpy ((char *) (result + *sz), temp[0].str, temp[0].len);
+      cp = __mempcpy (cp, temp[0].str, temp[0].len);
       *cp = '/';
-      *sz = 2;
-      *max_capstrlen = result[0].len;
+      if (result[0].len > hwcaps_counts.maximum_length)
+	*max_capstrlen = result[0].len;
+      else
+	*max_capstrlen = hwcaps_counts.maximum_length;
 
-      return result;
+      return overall_result;
     }
 
   /* Fill in the information.  This follows the following scheme
@@ -124,7 +223,7 @@ _dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
 	      #3: 0, 3			1001
      This allows the representation of all possible combinations of
      capability names in the string.  First generate the strings.  */
-  result[1].str = result[0].str = cp = (char *) (result + *sz);
+  result[1].str = result[0].str = cp;
 #define add(idx) \
       cp = __mempcpy (__mempcpy (cp, temp[idx].str, temp[idx].len), "/", 1);
   if (cnt == 2)
@@ -191,7 +290,10 @@ _dl_important_hwcaps (size_t *sz, size_t *max_capstrlen)
   while (--n != 0);
 
   /* The maximum string length.  */
-  *max_capstrlen = result[0].len;
+  if (result[0].len > hwcaps_counts.maximum_length)
+    *max_capstrlen = result[0].len;
+  else
+    *max_capstrlen = hwcaps_counts.maximum_length;
 
-  return result;
+  return overall_result;
 }
diff --git a/elf/dl-hwcaps.h b/elf/dl-hwcaps.h
index b66da59b89..a6453f15f3 100644
--- a/elf/dl-hwcaps.h
+++ b/elf/dl-hwcaps.h
@@ -16,6 +16,11 @@
    License along with the GNU C Library; if not, see
    <https://www.gnu.org/licenses/>.  */
 
+#ifndef _DL_HWCAPS_H
+#define _DL_HWCAPS_H
+
+#include <stdint.h>
+
 #include <elf/dl-tunables.h>
 
 #if HAVE_TUNABLES
@@ -28,3 +33,81 @@
 #  define GET_HWCAP_MASK() (0)
 # endif
 #endif
+
+#define GLIBC_HWCAPS_SUBDIRECTORY "glibc-hwcaps"
+#define GLIBC_HWCAPS_PREFIX GLIBC_HWCAPS_SUBDIRECTORY "/"
+
+/* Used by _dl_hwcaps_split below, to split strings at ':'
+   separators.  */
+struct dl_hwcaps_split
+{
+  const char *segment;          /* Start of the current segment.  */
+  size_t length;                /* Number of bytes until ':' or NUL.  */
+};
+
+/* Prepare *S to parse SUBJECT, for future _dl_hwcaps_split calls.  If
+   SUBJECT is NULL, it is treated as the empty string.  */
+static inline void
+_dl_hwcaps_split_init (struct dl_hwcaps_split *s, const char *subject)
+{
+  s->segment = subject;
+  /* The initial call to _dl_hwcaps_split will not skip anything.  */
+  s->length = 0;
+}
+
+/* Extract the next non-empty string segment, up to ':' or the null
+   terminator.  Return true if one more segment was found, or false if
+   the end of the string was reached.  On success, S->segment is the
+   start of the segment found, and S->length is its length.
+   (Typically, S->segment[S->length] is not null.)  */
+_Bool _dl_hwcaps_split (struct dl_hwcaps_split *s) attribute_hidden;
+
+/* Similar to dl_hwcaps_split, but with bit-based and name-based
+   masking.  */
+struct dl_hwcaps_split_masked
+{
+  struct dl_hwcaps_split split;
+
+  /* For used by the iterator implementation.  */
+  const char *mask;
+  int32_t bitmask;
+};
+
+/* Prepare *S for iteration with _dl_hwcaps_split_masked.  Only HWCAP
+   names in SUBJECT whose bit is set in BITMASK and whose ane is in
+   MASK will be returned.  SUBJECT must not contain empty HWCAP names.
+   If MASK is NULL, no name-based masking is applied.  Likewise for
+   BITMASK if BITMASK is -1 (infinite number of bits).  */
+static inline void
+_dl_hwcaps_split_masked_init (struct dl_hwcaps_split_masked *s,
+                              const char *subject,
+                              int32_t bitmask, const char *mask)
+{
+  _dl_hwcaps_split_init (&s->split, subject);
+  s->bitmask = bitmask;
+  s->mask = mask;
+}
+
+/* Like _dl_hwcaps_split, but apply masking.  */
+_Bool _dl_hwcaps_split_masked (struct dl_hwcaps_split_masked *s)
+  attribute_hidden;
+
+/* Returns true if the colon-separated HWCAP list HWCAPS contains the
+   capability NAME (with length NAME_LENGTH).  If HWCAPS is NULL, the
+   function returns true.  */
+_Bool _dl_hwcaps_contains (const char *hwcaps, const char *name,
+                           size_t name_length) attribute_hidden;
+
+/* Colon-separated string of glibc-hwcaps subdirectories, without the
+   "glibc-hwcaps/" prefix.  The most preferred subdirectory needs to
+   be listed first.  */
+extern const char _dl_hwcaps_subdirs[] attribute_hidden;
+
+/* Returns a bitmap of active subdirectories in _dl_hwcaps_subdirs.
+   Bit 0 (the LSB) corresponds to the first substring in
+   _dl_hwcaps_subdirs, bit 1 to the second substring, and so on.
+   There is no direct correspondence between HWCAP bitmasks and this
+   bitmask.  */
+int32_t _dl_hwcaps_subdirs_active (void) attribute_hidden;
+
+#endif /* _DL_HWCAPS_H */
diff --git a/elf/dl-hwcaps_split.c b/elf/dl-hwcaps_split.c
new file mode 100644
index 0000000000..95225e9f40
--- /dev/null
+++ b/elf/dl-hwcaps_split.c
@@ -0,0 +1,77 @@
+/* Hardware capability support for run-time dynamic loader.  String splitting.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <stdbool.h>
+#include <string.h>
+
+_Bool
+_dl_hwcaps_split (struct dl_hwcaps_split *s)
+{
+  if (s->segment == NULL)
+    return false;
+
+  /* Skip over the previous segment.   */
+  s->segment += s->length;
+
+  /* Consume delimiters.  This also avoids returning an empty
+     segment.  */
+  while (*s->segment == ':')
+    ++s->segment;
+  if (*s->segment == '\0')
+    return false;
+
+  /* This could use strchrnul, but we would have to link the function
+     into ld.so for that.  */
+  const char *colon = strchr (s->segment, ':');
+  if (colon == NULL)
+    s->length = strlen (s->segment);
+  else
+    s->length = colon - s->segment;
+  return true;
+}
+
+_Bool
+_dl_hwcaps_split_masked (struct dl_hwcaps_split_masked *s)
+{
+  while (true)
+    {
+      if (!_dl_hwcaps_split (&s->split))
+        return false;
+      bool active = s->bitmask & 1;
+      s->bitmask >>= 1;
+      if (active && _dl_hwcaps_contains (s->mask,
+                                         s->split.segment, s->split.length))
+        return true;
+    }
+}
+
+_Bool
+_dl_hwcaps_contains (const char *hwcaps, const char *name, size_t name_length)
+{
+  if (hwcaps == NULL)
+    return true;
+
+  struct dl_hwcaps_split split;
+  _dl_hwcaps_split_init (&split, hwcaps);
+  while (_dl_hwcaps_split (&split))
+    if (split.length == name_length
+        && memcmp (split.segment, name, name_length) == 0)
+      return true;
+  return false;
+}
diff --git a/elf/dl-load.c b/elf/dl-load.c
index 6e5fa2af13..f99d791edb 100644
--- a/elf/dl-load.c
+++ b/elf/dl-load.c
@@ -682,7 +682,9 @@ cache_rpath (struct link_map *l,
 
 
 void
-_dl_init_paths (const char *llp, const char *source)
+_dl_init_paths (const char *llp, const char *source,
+		const char *glibc_hwcaps_prepend,
+		const char *glibc_hwcaps_mask)
 {
   size_t idx;
   const char *strp;
@@ -697,7 +699,8 @@ _dl_init_paths (const char *llp, const char *source)
 
 #ifdef SHARED
   /* Get the capabilities.  */
-  capstr = _dl_important_hwcaps (&ncapstr, &max_capstrlen);
+  capstr = _dl_important_hwcaps (glibc_hwcaps_prepend, glibc_hwcaps_mask,
+				 &ncapstr, &max_capstrlen);
 #endif
 
   /* First set up the rest of the default search directory entries.  */
diff --git a/elf/dl-main.h b/elf/dl-main.h
index 0df849d3cd..710d29685b 100644
--- a/elf/dl-main.h
+++ b/elf/dl-main.h
@@ -80,6 +80,14 @@ struct dl_main_state
   /* The preload list passed as a command argument.  */
   const char *preloadarg;
 
+  /* Additional glibc-hwcaps subdirectories to search first.
+     Colon-separated list.  */
+  const char *glibc_hwcaps_prepend;
+
+  /* Mask for the internal glibc-hwcaps subdirectories.
+     Colon-separated list.  */
+  const char *glibc_hwcaps_mask;
+
   enum mode mode;
 
   /* True if any of the debugging options is enabled.  */
@@ -94,7 +102,8 @@ struct dl_main_state
 static inline void
 call_init_paths (const struct dl_main_state *state)
 {
-  _dl_init_paths (state->library_path, state->library_path_source);
+  _dl_init_paths (state->library_path, state->library_path_source,
+                  state->glibc_hwcaps_prepend, state->glibc_hwcaps_mask);
 }
 
 /* Print ld.so usage information and exit.  */
diff --git a/elf/dl-support.c b/elf/dl-support.c
index afbc94df54..3264262f4e 100644
--- a/elf/dl-support.c
+++ b/elf/dl-support.c
@@ -323,7 +323,10 @@ _dl_non_dynamic_init (void)
 
   /* Initialize the data structures for the search paths for shared
      objects.  */
-  _dl_init_paths (getenv ("LD_LIBRARY_PATH"), "LD_LIBRARY_PATH");
+  _dl_init_paths (getenv ("LD_LIBRARY_PATH"), "LD_LIBRARY_PATH",
+		  /* No glibc-hwcaps selection support in statically
+		     linked binaries.  */
+		  NULL, NULL);
 
   /* Remember the last search directory added at startup.  */
   _dl_init_all_dirs = GL(dl_all_dirs);
diff --git a/elf/dl-usage.c b/elf/dl-usage.c
index 7ff642349c..e94334e877 100644
--- a/elf/dl-usage.c
+++ b/elf/dl-usage.c
@@ -82,7 +82,7 @@ print_search_path_for_help (struct dl_main_state *state)
 {
   if (__rtld_search_dirs.dirs == NULL)
     /* The run-time search paths have not yet been initialized.  */
-    _dl_init_paths (state->library_path, state->library_path_source);
+    call_init_paths (state);
 
   _dl_printf ("\nShared library search path:\n");
 
@@ -131,6 +131,67 @@ print_hwcap_1_finish (bool *first)
     _dl_printf (")\n");
 }
 
+/* Print the header for print_hwcaps_subdirectories.  */
+static void
+print_hwcaps_subdirectories_header (bool *nothing_printed)
+{
+  if (*nothing_printed)
+    {
+      _dl_printf ("\n\
+Subdirectories of glibc-hwcaps directories, in priority order:\n");
+      *nothing_printed = false;
+    }
+}
+
+/* Print the HWCAP name itself, indented.  */
+static void
+print_hwcaps_subdirectories_name (const struct dl_hwcaps_split *split)
+{
+  _dl_write (STDOUT_FILENO, "  ", 2);
+  _dl_write (STDOUT_FILENO, split->segment, split->length);
+}
+
+/* Print the list of recognized glibc-hwcaps subdirectories.  */
+static void
+print_hwcaps_subdirectories (const struct dl_main_state *state)
+{
+  bool nothing_printed = true;
+  struct dl_hwcaps_split split;
+
+  /* The prepended glibc-hwcaps subdirectories.  */
+  _dl_hwcaps_split_init (&split, state->glibc_hwcaps_prepend);
+  while (_dl_hwcaps_split (&split))
+    {
+      print_hwcaps_subdirectories_header (&nothing_printed);
+      print_hwcaps_subdirectories_name (&split);
+      bool first = true;
+      print_hwcap_1 (&first, true, "searched");
+      print_hwcap_1_finish (&first);
+    }
+
+  /* The built-in glibc-hwcaps subdirectories.  Do the filtering
+     manually, so that more precise diagnostics are possible.  */
+  int32_t mask = _dl_hwcaps_subdirs_active ();
+  _dl_hwcaps_split_init (&split, _dl_hwcaps_subdirs);
+  while (_dl_hwcaps_split (&split))
+    {
+      print_hwcaps_subdirectories_header (&nothing_printed);
+      print_hwcaps_subdirectories_name (&split);
+      bool first = true;
+      print_hwcap_1 (&first, mask & 1, "supported");
+      bool listed = _dl_hwcaps_contains (state->glibc_hwcaps_mask,
+                                         split.segment, split.length);
+      print_hwcap_1 (&first, !listed, "masked");
+      print_hwcap_1 (&first, (mask & 1) && listed, "searched");
+      print_hwcap_1_finish (&first);
+      mask >>= 1;
+    }
+
+  if (nothing_printed)
+    _dl_printf ("\n\
+No subdirectories of glibc-hwcaps directories are searched.\n");
+}
+
 /* Write a list of hwcap subdirectories to standard output.  See
  _dl_important_hwcaps in dl-hwcaps.c.  */
 static void
@@ -185,6 +246,10 @@ setting environment variables (which would be inherted by subprocesses).\n\
   --inhibit-cache       Do not use " LD_SO_CACHE "\n\
   --library-path PATH   use given PATH instead of content of the environment\n\
                         variable LD_LIBRARY_PATH\n\
+  --glibc-hwcaps-prepend LIST\n\
+                        search glibc-hwcaps subdirectories in LIST\n\
+  --glibc-hwcaps-mask LIST\n\
+                        only search built-in subdirectories if in LIST\n\
   --inhibit-rpath LIST  ignore RUNPATH and RPATH information in object names\n\
                         in LIST\n\
   --audit LIST          use objects named in LIST as auditors\n\
@@ -196,6 +261,7 @@ This program interpreter self-identifies as: " RTLD "\n\
 ",
               argv0);
   print_search_path_for_help (state);
+  print_hwcaps_subdirectories (state);
   print_legacy_hwcap_directories ();
   _exit (0);
 }
diff --git a/elf/rtld.c b/elf/rtld.c
index 06bc8eca9a..b31597da3c 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -272,6 +272,8 @@ dl_main_state_init (struct dl_main_state *state)
   state->library_path_source = NULL;
   state->preloadlist = NULL;
   state->preloadarg = NULL;
+  state->glibc_hwcaps_prepend = NULL;
+  state->glibc_hwcaps_mask = NULL;
   state->mode = normal;
   state->any_debug = false;
   state->version_info = false;
@@ -1218,6 +1220,22 @@ dl_main (const ElfW(Phdr) *phdr,
 	    _dl_argc -= 2;
 	    _dl_argv += 2;
 	  }
+	else if (strcmp (_dl_argv[1], "--glibc-hwcaps-prepend") == 0
+		 && _dl_argc > 2)
+	  {
+	    state.glibc_hwcaps_prepend = _dl_argv[2];
+	    _dl_skip_args += 2;
+	    _dl_argc -= 2;
+	    _dl_argv += 2;
+	  }
+	else if (strcmp (_dl_argv[1], "--glibc-hwcaps-mask") == 0
+		 && _dl_argc > 2)
+	  {
+	    state.glibc_hwcaps_mask = _dl_argv[2];
+	    _dl_skip_args += 2;
+	    _dl_argc -= 2;
+	    _dl_argv += 2;
+	  }
 	else if (strcmp (_dl_argv[1], "--help") == 0)
 	  {
 	    state.mode = rtld_help;
diff --git a/elf/tst-dl-hwcaps_split.c b/elf/tst-dl-hwcaps_split.c
new file mode 100644
index 0000000000..929c99a23b
--- /dev/null
+++ b/elf/tst-dl-hwcaps_split.c
@@ -0,0 +1,139 @@
+/* Unit tests for dl-hwcaps.c.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <array_length.h>
+#include <dl-hwcaps.h>
+#include <string.h>
+#include <support/check.h>
+
+static void
+check_split_masked (const char *input, int32_t bitmask, const char *mask,
+                    const char *expected[], size_t expected_length)
+{
+  struct dl_hwcaps_split_masked split;
+  _dl_hwcaps_split_masked_init (&split, input, bitmask, mask);
+  size_t index = 0;
+  while (_dl_hwcaps_split_masked (&split))
+    {
+      TEST_VERIFY_EXIT (index < expected_length);
+      TEST_COMPARE_BLOB (expected[index], strlen (expected[index]),
+                         split.split.segment, split.split.length);
+      ++index;
+    }
+  TEST_COMPARE (index, expected_length);
+}
+
+static void
+check_split (const char *input,
+             const char *expected[], size_t expected_length)
+{
+  struct dl_hwcaps_split split;
+  _dl_hwcaps_split_init (&split, input);
+  size_t index = 0;
+  while (_dl_hwcaps_split (&split))
+    {
+      TEST_VERIFY_EXIT (index < expected_length);
+      TEST_COMPARE_BLOB (expected[index], strlen (expected[index]),
+                         split.segment, split.length);
+      ++index;
+    }
+  TEST_COMPARE (index, expected_length);
+
+  /* Reuse the test cases with masking that does not actually remove
+     anything.  */
+  check_split_masked (input, -1, NULL, expected, expected_length);
+  check_split_masked (input, -1, input, expected, expected_length);
+}
+
+static int
+do_test (void)
+{
+  /* Splitting tests, without masking.  */
+  check_split (NULL, NULL, 0);
+  check_split ("", NULL, 0);
+  check_split (":", NULL, 0);
+  check_split ("::", NULL, 0);
+
+  {
+    const char *expected[] = { "first" };
+    check_split ("first", expected, array_length (expected));
+    check_split (":first", expected, array_length (expected));
+    check_split ("first:", expected, array_length (expected));
+    check_split (":first:", expected, array_length (expected));
+  }
+
+  {
+    const char *expected[] = { "first", "second" };
+    check_split ("first:second", expected, array_length (expected));
+    check_split ("first::second", expected, array_length (expected));
+    check_split (":first:second", expected, array_length (expected));
+    check_split ("first:second:", expected, array_length (expected));
+    check_split (":first:second:", expected, array_length (expected));
+  }
+
+  /* Splitting tests with masking.  */
+  {
+    const char *expected[] = { "first" };
+    check_split_masked ("first", 3, "first:second",
+                        expected, array_length (expected));
+    check_split_masked ("first:second", 3, "first:",
+                        expected, array_length (expected));
+    check_split_masked ("first:second", 1, NULL,
+                        expected, array_length (expected));
+  }
+  {
+    const char *expected[] = { "second" };
+    check_split_masked ("first:second", 3, "second",
+                        expected, array_length (expected));
+    check_split_masked ("first:second:third", -1, "second:",
+                        expected, array_length (expected));
+    check_split_masked ("first:second", 2, NULL,
+                        expected, array_length (expected));
+    check_split_masked ("first:second:third", 2, "first:second",
+                        expected, array_length (expected));
+  }
+
+  /* Tests for _dl_hwcaps_contains.  */
+  TEST_VERIFY (_dl_hwcaps_contains (NULL, "first", strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains (NULL, "", 0));
+  TEST_VERIFY (! _dl_hwcaps_contains ("", "first", strlen ("first")));
+  TEST_VERIFY (! _dl_hwcaps_contains ("firs", "first", strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains ("firs", "first", strlen ("first") - 1));
+  for (int i = 0; i < strlen ("first"); ++i)
+    TEST_VERIFY (! _dl_hwcaps_contains ("first", "first", i));
+  TEST_VERIFY (_dl_hwcaps_contains ("first", "first", strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains ("first:", "first", strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains ("first:second",
+                                    "first", strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains (":first:second", "first",
+                                    strlen ("first")));
+  TEST_VERIFY (_dl_hwcaps_contains ("first:second", "second",
+                                    strlen ("second")));
+  TEST_VERIFY (_dl_hwcaps_contains ("first:second:", "second",
+                                    strlen ("second")));
+  for (int i = 0; i < strlen ("second"); ++i)
+    TEST_VERIFY (!_dl_hwcaps_contains ("first:second:", "sec", i));
+
+  return 0;
+}
+
+#include <support/test-driver.c>
+
+/* Rebuild the sources here because the object file is built for
+   inclusion into the dynamic loader.  */
+#include "dl-hwcaps_split.c"
diff --git a/sysdeps/generic/ldsodefs.h b/sysdeps/generic/ldsodefs.h
index 2de060848b..7e14650422 100644
--- a/sysdeps/generic/ldsodefs.h
+++ b/sysdeps/generic/ldsodefs.h
@@ -1035,8 +1035,13 @@ extern struct r_debug *_dl_debug_initialize (ElfW(Addr) ldbase, Lmid_t ns)
      attribute_hidden;
 
 /* Initialize the basic data structure for the search paths.  SOURCE
-   is either "LD_LIBRARY_PATH" or "--library-path".  */
-extern void _dl_init_paths (const char *library_path, const char *source)
+   is either "LD_LIBRARY_PATH" or "--library-path".
+   GLIBC_HWCAPS_PREPEND adds additional glibc-hwcaps subdirectories to
+   search.  GLIBC_HWCAPS_MASK is used to filter the built-in
+   subdirectories if not NULL.  */
+extern void _dl_init_paths (const char *library_path, const char *source,
+			    const char *glibc_hwcaps_prepend,
+			    const char *glibc_hwcaps_mask)
   attribute_hidden;
 
 /* Gather the information needed to install the profiling tables and start
@@ -1060,9 +1065,14 @@ extern void _dl_show_auxv (void) attribute_hidden;
 extern char *_dl_next_ld_env_entry (char ***position) attribute_hidden;
 
 /* Return an array with the names of the important hardware
-   capabilities.  The length of the array is written to *SZ, and the
-   maximum of all strings length is written to *MAX_CAPSTRLEN.  */
-const struct r_strlenpair *_dl_important_hwcaps (size_t *sz,
+   capabilities.  PREPEND is a colon-separated list of glibc-hwcaps
+   directories to search first.  MASK is a colon-separated list used
+   to filter the built-in glibc-hwcaps subdirectories.  The length of
+   the array is written to *SZ, and the maximum of all strings length
+   is written to *MAX_CAPSTRLEN.  */
+const struct r_strlenpair *_dl_important_hwcaps (const char *prepend,
+						 const char *mask,
+						 size_t *sz,
 						 size_t *max_capstrlen)
   attribute_hidden;
 
-- 
2.25.4

[PATCH 19/30] x86_64: Add glibc-hwcaps support

[Florian Weimer]

Details of the supported CPU flags and the names are still subject to
changes.
---
 sysdeps/x86_64/dl-hwcaps-subdirs.c | 73 ++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 sysdeps/x86_64/dl-hwcaps-subdirs.c

diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..4a8fae976e
--- /dev/null
+++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
@@ -0,0 +1,73 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  x86 version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <cpu-features.h>
+
+const char _dl_hwcaps_subdirs[] = "x86-103:x86-102:x86-101:x86-100";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  const struct cpu_features *cpu_features = __get_cpu_features ();
+  int32_t result = 0;
+  int32_t bit = 1 << 3;
+
+  /* Test in reverse preference order.  */
+
+  /* x86-100.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, CMPXCHG16B_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, POPCNT_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, SSE3_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_1_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_2_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, SSSE3_Usable)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* x86-101.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX_Usable)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* x86-102.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, BMI1_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, BMI2_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, F16C_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, LZCNT_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, MOVBE_Usable)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+ /* x86-103.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX512F_Usable)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512BW)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512CD)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512DQ)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512VL)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  return result;
+}
-- 
2.25.4

[PATCH 20/30] powerpc64le: Add glibc-hwcaps support

[Florian Weimer]

At this time, the only selected subdirectory is "power9", for systems
with ISA 3.00 support.
---
 .../powerpc/powerpc64/le/dl-hwcaps-subdirs.c  | 31 +++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c

diff --git a/sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c b/sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..fab5d2b8fb
--- /dev/null
+++ b/sysdeps/powerpc/powerpc64/le/dl-hwcaps-subdirs.c
@@ -0,0 +1,31 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  powerpc64le version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <ldsodefs.h>
+
+const char _dl_hwcaps_subdirs[] = "power9";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  if (GLRO (dl_hwcap2) & PPC_FEATURE2_ARCH_3_00)
+    return 1;
+
+  return 0;
+}
-- 
2.25.4

[PATCH 21/30] s390x: Add Add glibc-hwcaps support

[Florian Weimer]

Subdirectories z13, z14, z15 can be selected, mostly based on the
level of support for vector instructions.
---
 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c | 54 ++++++++++++++++++++++++
 1 file changed, 54 insertions(+)
 create mode 100644 sysdeps/s390/s390-64/dl-hwcaps-subdirs.c

diff --git a/sysdeps/s390/s390-64/dl-hwcaps-subdirs.c b/sysdeps/s390/s390-64/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..255af22f02
--- /dev/null
+++ b/sysdeps/s390/s390-64/dl-hwcaps-subdirs.c
@@ -0,0 +1,54 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  s390x version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <ldsodefs.h>
+
+const char _dl_hwcaps_subdirs[] = "z15:z14:z13";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  int32_t result = 0;
+  int32_t bit = 1 << 2;
+
+  /* Test in reverse preference order.  */
+
+  /* z13.  */
+  if (!(GLRO (dl_hwcap) & HWCAP_S390_VX))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* z14.  */
+  if (!((GLRO (dl_hwcap) & HWCAP_S390_VXD)
+        && (GLRO (dl_hwcap) & HWCAP_S390_VXE)
+        && (GLRO (dl_hwcap) & HWCAP_S390_GS)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* z15.  */
+  if (!((GLRO (dl_hwcap) & HWCAP_S390_VXRS_EXT2)
+        && (GLRO (dl_hwcap) & HWCAP_S390_VXRS_PDE)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  return result;
+}
-- 
2.25.4

[PATCH 22/30] aarch64: Add glibc-hwcaps support

[Florian Weimer]

At this point, only the "atomics" subdirectory is available,
for libraries built using LSE atomics.
---
 sysdeps/aarch64/dl-hwcaps-subdirs.c | 31 +++++++++++++++++++++++++++++
 1 file changed, 31 insertions(+)
 create mode 100644 sysdeps/aarch64/dl-hwcaps-subdirs.c

diff --git a/sysdeps/aarch64/dl-hwcaps-subdirs.c b/sysdeps/aarch64/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..fd6325024e
--- /dev/null
+++ b/sysdeps/aarch64/dl-hwcaps-subdirs.c
@@ -0,0 +1,31 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  aarch64 version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <ldsodefs.h>
+
+const char _dl_hwcaps_subdirs[] = "atomics";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  if (GLRO (dl_hwcap) & HWCAP_ATOMICS)
+    return 1;
+
+  return 0;
+}
-- 
2.25.4

[PATCH 23/30] elf: Add endianness markup to ld.so.cache

[Florian Weimer]

Use a reserved byte in the new format cache header to indicate whether
the file is in little endian or big endian format.  Eventually, this
information could be used to provide a unified cache for qemu-user
and similiar scenarios.
---
 elf/cache.c                | 11 ++++++++++
 elf/dl-cache.c             | 20 +++++++++++++++++-
 sysdeps/generic/dl-cache.h | 43 +++++++++++++++++++++++++++++++++++++-
 3 files changed, 72 insertions(+), 2 deletions(-)

diff --git a/elf/cache.c b/elf/cache.c
index d92b4e59c1..c10981641f 100644
--- a/elf/cache.c
+++ b/elf/cache.c
@@ -152,6 +152,14 @@ print_entry (const char *lib, int flag, unsigned int osversion,
   printf (") => %s\n", key);
 }
 
+/* Print an error and exit if the new-file cache is internally
+   inconsistent.  */
+static void
+check_new_cache (struct cache_file_new *cache)
+{
+  if (! cache_file_new_matches_endian (cache))
+    error (EXIT_FAILURE, 0, _("Cache file has wrong endianness.\n"));
+}
 
 /* Print the whole cache file, if a file contains the new cache format
    hidden in the old one, print the contents of the new format.  */
@@ -193,6 +201,7 @@ print_cache (const char *cache_name)
 	  || memcmp (cache_new->version, CACHE_VERSION,
 		      sizeof CACHE_VERSION - 1))
 	error (EXIT_FAILURE, 0, _("File is not a cache file.\n"));
+      check_new_cache (cache_new);
       format = 1;
       /* This is where the strings start.  */
       cache_data = (const char *) cache_new;
@@ -222,6 +231,7 @@ print_cache (const char *cache_name)
 	      && memcmp (cache_new->version, CACHE_VERSION,
 			 sizeof CACHE_VERSION - 1) == 0)
 	    {
+	      check_new_cache (cache_new);
 	      cache_data = (const char *) cache_new;
 	      format = 1;
 	    }
@@ -361,6 +371,7 @@ save_cache (const char *cache_name)
 
       file_entries_new->nlibs = cache_entry_count;
       file_entries_new->len_strings = total_strlen;
+      file_entries_new->flags = cache_file_new_flags_endian;
     }
 
   /* Pad for alignment of cache_file_new.  */
diff --git a/elf/dl-cache.c b/elf/dl-cache.c
index 93d185e788..3aa8ed6c13 100644
--- a/elf/dl-cache.c
+++ b/elf/dl-cache.c
@@ -210,6 +210,11 @@ _dl_load_cache_lookup (const char *name)
 	  && ((cachesize - sizeof *cache_new) / sizeof (struct file_entry_new)
 	      >= ((struct cache_file_new *) file)->nlibs))
 	{
+	  if (! cache_file_new_matches_endian (file))
+	    {
+	      __munmap (file, cachesize);
+	      file = (void *) -1;
+	    }
 	  cache_new = file;
 	  cache = file;
 	}
@@ -231,7 +236,20 @@ _dl_load_cache_lookup (const char *name)
 	  if (cachesize < (offset + sizeof (struct cache_file_new))
 	      || memcmp (cache_new->magic, CACHEMAGIC_VERSION_NEW,
 			 sizeof CACHEMAGIC_VERSION_NEW - 1) != 0)
-	    cache_new = (void *) -1;
+	      cache_new = (void *) -1;
+	  else
+	    {
+	      if (! cache_file_new_matches_endian (cache_new))
+		{
+		  /* The old-format part of the cache is bogus as well
+		     if the endianness does not match.  (But it is
+		     unclear how the new header can be located if the
+		     endianess does not match.)  */
+		  cache = (void *) -1;
+		  cache_new = (void *) -1;
+		  __munmap (file, cachesize);
+		}
+	    }
 	}
       else
 	{
diff --git a/sysdeps/generic/dl-cache.h b/sysdeps/generic/dl-cache.h
index 6b310e9e15..1b04211f6b 100644
--- a/sysdeps/generic/dl-cache.h
+++ b/sysdeps/generic/dl-cache.h
@@ -16,6 +16,11 @@
    License along with the GNU C Library; if not, see
    <https://www.gnu.org/licenses/>.  */
 
+#ifndef _DL_CACHE_H
+#define _DL_CACHE_H
+
+#include <endian.h>
+#include <stdbool.h>
 #include <stdint.h>
 
 #ifndef _DL_CACHE_DEFAULT_ID
@@ -83,21 +88,57 @@ struct file_entry_new
   uint64_t hwcap;		/* Hwcap entry.	 */
 };
 
+/* See flags member of struct cache_file_new below.  */
+enum
+  {
+   cache_file_new_flags_endian = (__BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
+				  ? 2 : 3)
+  };
+
 struct cache_file_new
 {
   char magic[sizeof CACHEMAGIC_NEW - 1];
   char version[sizeof CACHE_VERSION - 1];
   uint32_t nlibs;		/* Number of entries.  */
   uint32_t len_strings;		/* Size of string table. */
-  uint32_t unused[5];		/* Leave space for future extensions
+
+  /* flags & 3 is used to indicate the endianness of the cache.
+     0: no endianness information available
+        (An old ldconfig version without endianness support wrote the file.)
+     1: cache is invalid
+     2: little endian
+     3: big endian
+
+     The remaining bits are unused and should be generated as zero and
+     ignored by readers.  */
+  uint8_t flags;
+
+  uint8_t padding_unsed[3];	/* Not used, for future extensions.  */
+
+  uint32_t unused[4];		/* Leave space for future extensions
 				   and align to 8 byte boundary.  */
   struct file_entry_new libs[0]; /* Entries describing libraries.  */
   /* After this the string table of size len_strings is found.	*/
 };
 
+/* Returns false if *CACHE has the wrong endianness for this
+   architecture, and true if the endianness matches (or is
+   unknown).  */
+static inline bool
+cache_file_new_matches_endian (const struct cache_file_new *cache)
+{
+  /* A zero value for cache->flags means that no endianness
+     information is available.  */
+  return cache->flags == 0
+    || (cache->flags & 3) == cache_file_new_flags_endian;
+}
+
+
 /* Used to align cache_file_new.  */
 #define ALIGN_CACHE(addr)				\
 (((addr) + __alignof__ (struct cache_file_new) -1)	\
  & (~(__alignof__ (struct cache_file_new) - 1)))
 
 extern int _dl_cache_libcmp (const char *p1, const char *p2) attribute_hidden;
+
+#endif /* _DL_CACHE_H */
-- 
2.25.4

[PATCH 24/30] elf: Add extension mechanism to ld.so.cache

[Florian Weimer]

A previously unused new-format header field is used to record
the address of an extension directory.

This change adds a demo extension which records the version of
ldconfig which builds a file.
---
 elf/cache.c                |  89 +++++++++++++++++++++++++++
 sysdeps/generic/dl-cache.h | 123 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 211 insertions(+), 1 deletion(-)

diff --git a/elf/cache.c b/elf/cache.c
index c10981641f..ee4d74fed1 100644
--- a/elf/cache.c
+++ b/elf/cache.c
@@ -15,6 +15,7 @@
    You should have received a copy of the GNU General Public License
    along with this program; if not, see <https://www.gnu.org/licenses/>.  */
 
+#include <assert.h>
 #include <errno.h>
 #include <error.h>
 #include <dirent.h>
@@ -33,6 +34,7 @@
 
 #include <ldconfig.h>
 #include <dl-cache.h>
+#include <version.h>
 
 struct cache_entry
 {
@@ -161,6 +163,21 @@ check_new_cache (struct cache_file_new *cache)
     error (EXIT_FAILURE, 0, _("Cache file has wrong endianness.\n"));
 }
 
+/* Print the extension information at the cache at start address
+   FILE_BASE, of ltength FILE_SIZE bytes.  The new-format cache header
+   is at CACHE, and the file name for diagnostics is CACHE_NAME.  */
+static void
+print_extensions (struct cache_extension_all_loaded *ext)
+{
+  if (ext->sections[cache_extension_tag_generator].base != NULL)
+    {
+      fputs (_("Cache generated by: "), stdout);
+      fwrite (ext->sections[cache_extension_tag_generator].base, 1,
+	      ext->sections[cache_extension_tag_generator].size, stdout);
+      putchar ('\n');
+    }
+}
+
 /* Print the whole cache file, if a file contains the new cache format
    hidden in the old one, print the contents of the new format.  */
 void
@@ -250,6 +267,11 @@ print_cache (const char *cache_name)
     }
   else if (format == 1)
     {
+      struct cache_extension_all_loaded ext;
+      if (!cache_extension_load (cache_new, cache, cache_size, &ext))
+	error (EXIT_FAILURE, 0,
+	       _("Malformed extension data in cache file %s\n"), cache_name);
+
       printf (_("%d libs found in cache `%s'\n"),
 	      cache_new->nlibs, cache_name);
 
@@ -260,6 +282,7 @@ print_cache (const char *cache_name)
 		     cache_new->libs[i].osversion,
 		     cache_new->libs[i].hwcap,
 		     cache_data + cache_new->libs[i].value);
+      print_extensions (&ext);
     }
   /* Cleanup.  */
   munmap (cache, cache_size);
@@ -301,6 +324,45 @@ compare (const struct cache_entry *e1, const struct cache_entry *e2)
   return res;
 }
 
+/* Size of the cache extension directory.  All tags are assumed to be
+   present.  */
+enum
+  {
+   cache_extension_size = (offsetof (struct cache_extension, sections)
+			   + (cache_extension_count
+			      * sizeof (struct cache_extension_section)))
+  };
+
+/* Write the cache extensions to FD.  The extension directory is
+   assumed to be located at CACHE_EXTENSION_OFFSET.  */
+static void
+write_extensions (int fd, uint32_t cache_extension_offset)
+{
+  assert ((cache_extension_offset % 4) == 0);
+
+  struct cache_extension *ext = xmalloc (cache_extension_size);
+  ext->magic = cache_extension_magic;
+  ext->count = cache_extension_count;
+
+  for (int i = 0; i < cache_extension_count; ++i)
+    {
+      ext->sections[i].tag = i;
+      ext->sections[i].flags = 0;
+    }
+
+  const char *generator
+    = "ldconfig " PKGVERSION RELEASE " release version " VERSION;
+  ext->sections[cache_extension_tag_generator].offset
+    = cache_extension_offset + cache_extension_size;
+  ext->sections[cache_extension_tag_generator].size = strlen (generator);
+
+  if (write (fd, ext, cache_extension_size) != cache_extension_size
+      || write (fd, generator, strlen (generator)) != strlen (generator))
+    error (EXIT_FAILURE, errno, _("Writing of cache extension data failed"));
+
+  free (ext);
+}
+
 /* Save the contents of the cache.  */
 void
 save_cache (const char *cache_name)
@@ -435,6 +497,25 @@ save_cache (const char *cache_name)
       && idx_old < cache_entry_old_count)
     file_entries->libs[idx_old] = file_entries->libs[idx_old - 1];
 
+  /* Compute the location of the extension directory.  This
+     implementation puts the directory after the string table.  The
+     size computation matches the write calls below.  The extension
+     directory does not exist with format 0, so the value does not
+     matter.  */
+  uint32_t extension_offset = 0;
+  if (opt_format != 2)
+    extension_offset += file_entries_size;
+  if (opt_format != 0)
+    {
+      if (opt_format != 2)
+	extension_offset += pad;
+      extension_offset += file_entries_new_size;
+    }
+  extension_offset += total_strlen;
+  extension_offset = roundup (extension_offset, 4); /* Provide alignment.  */
+  if (opt_format != 0)
+    file_entries_new->extension_offset = extension_offset;
+
   /* Write out the cache.  */
 
   /* Write cache first to a temporary file and rename it later.  */
@@ -473,6 +554,14 @@ save_cache (const char *cache_name)
   if (write (fd, strings, total_strlen) != (ssize_t) total_strlen)
     error (EXIT_FAILURE, errno, _("Writing of cache data failed"));
 
+  if (opt_format != 0)
+    {
+      /* Align file position to 4.  */
+      off64_t old_offset = lseek64 (fd, extension_offset, SEEK_SET);
+      assert ((unsigned long long int) (extension_offset - old_offset) < 4);
+      write_extensions (fd, extension_offset);
+    }
+
   /* Make sure user can always read cache file */
   if (chmod (temp_name, S_IROTH|S_IRGRP|S_IRUSR|S_IWUSR))
     error (EXIT_FAILURE, errno,
diff --git a/sysdeps/generic/dl-cache.h b/sysdeps/generic/dl-cache.h
index 1b04211f6b..b154740da9 100644
--- a/sysdeps/generic/dl-cache.h
+++ b/sysdeps/generic/dl-cache.h
@@ -21,7 +21,9 @@
 
 #include <endian.h>
 #include <stdbool.h>
+#include <stddef.h>
 #include <stdint.h>
+#include <string.h>
 
 #ifndef _DL_CACHE_DEFAULT_ID
 # define _DL_CACHE_DEFAULT_ID	3
@@ -115,7 +117,11 @@ struct cache_file_new
 
   uint8_t padding_unsed[3];	/* Not used, for future extensions.  */
 
-  uint32_t unused[4];		/* Leave space for future extensions
+  /* File offset of the extension directory.  See struct
+     cache_extension below.  Must be a multiple of four.  */
+  uint32_t extension_offset;
+
+  uint32_t unused[3];		/* Leave space for future extensions
 				   and align to 8 byte boundary.  */
   struct file_entry_new libs[0]; /* Entries describing libraries.  */
   /* After this the string table of size len_strings is found.	*/
@@ -134,6 +140,121 @@ cache_file_new_matches_endian (const struct cache_file_new *cache)
 }
 
 
+/* Randomly chosen magic value, which allows for additional
+   consistency verification.  */
+enum { cache_extension_magic = (uint32_t) -358342284 };
+
+/* Tag values for different kinds of extension sections.  Similar to
+   SHT_* constants.  */
+enum cache_extension_tag
+  {
+   /* Array of bytes containing the glibc version that generated this
+      cache file.  */
+   cache_extension_tag_generator,
+
+   /* Total number of known cache extension tags.  */
+   cache_extension_count
+  };
+
+/* Element in the array following struct cache_extension.  Similar to
+   an ELF section header.  */
+struct cache_extension_section
+{
+  /* Type of the extension section.  A enum cache_extension_tag value.  */
+  uint32_t tag;
+
+  /* Extension-specific flags.  Currently generated as zero.  */
+  uint32_t flags;
+
+  /* Offset from the start of the file for the data in this extension
+     section.  Specific extensions can have alignment constraints.  */
+  uint32_t offset;
+
+  /* Length in bytes of the extension data.  Specific extensions may
+     have size requirements.  */
+  uint32_t size;
+};
+
+/* The extension directory in the cache.  An array of struct
+   cache_extension_section entries.  */
+struct cache_extension
+{
+  uint32_t magic;		/* Always cache_extension_magic.  */
+  uint32_t count;		/* Number of following entries.  */
+
+  /* count section descriptors of type struct cache_extension_section
+     follow.  */
+  struct cache_extension_section sections[];
+};
+
+/* A relocated version of struct cache_extension_section.  */
+struct cache_extension_loaded
+{
+  /* Address and size of this extension section.  base is NULL if the
+     section is missing from the file.  */
+  const void *base;
+  size_t size;
+
+  /* Flags from struct cache_extension_section.  */
+  uint32_t flags;
+};
+
+/* All supported extension sections, relocated.  Filled in by
+   cache_extension_load below.  */
+struct cache_extension_all_loaded
+{
+  struct cache_extension_loaded sections[cache_extension_count];
+};
+
+static bool __attribute__ ((unused))
+cache_extension_load (const struct cache_file_new *cache,
+		      const void *file_base, size_t file_size,
+		      struct cache_extension_all_loaded *loaded)
+{
+  memset (loaded, 0, sizeof (*loaded));
+  if (cache->extension_offset == 0)
+    /* No extensions present.  This is not a format error.  */
+    return true;
+  if ((cache->extension_offset % 4) != 0)
+    /* Extension offset is misaligned.  */
+    return false;
+  size_t size_tmp;
+  if (__builtin_add_overflow (cache->extension_offset,
+			      sizeof (struct cache_extension), &size_tmp)
+      || size_tmp > file_size)
+    /* Extension extends beyond the end of the file.  */
+    return false;
+  const struct cache_extension *ext = file_base + cache->extension_offset;
+  if (ext->magic != cache_extension_magic)
+    return false;
+  if (__builtin_mul_overflow (ext->count,
+			      sizeof (struct cache_extension_section),
+			      &size_tmp)
+      || __builtin_add_overflow (cache->extension_offset
+				 + sizeof (struct cache_extension), size_tmp,
+				 &size_tmp)
+      || size_tmp > file_size)
+    /* Extension array extends beyond the end of the file.  */
+    return false;
+  for (uint32_t i = 0; i < ext->count; ++i)
+    {
+      if (__builtin_add_overflow (ext->sections[i].offset,
+				  ext->sections[i].size, &size_tmp)
+	  || size_tmp > file_size)
+	/* Extension data extends beyond the end of the file.  */
+	return false;
+
+      uint32_t tag = ext->sections[i].tag;
+      if (tag >= cache_extension_count)
+	/* Tag is out of range and unrecognized.  */
+	continue;
+      loaded->sections[tag].base = file_base + ext->sections[i].offset;
+      loaded->sections[tag].size = ext->sections[i].size;
+      loaded->sections[tag].flags = ext->sections[i].flags;
+    }
+  return true;
+}
+
 /* Used to align cache_file_new.  */
 #define ALIGN_CACHE(addr)				\
 (((addr) + __alignof__ (struct cache_file_new) -1)	\
-- 
2.25.4

[PATCH 25/30] elf: Unify old and new format cache handling code in ld.so

[Florian Weimer]

struct file_entry_new starts with the fields of struct file_entry,
so the code can be shared if the size computation is made dynamic.
---
 elf/dl-cache.c             | 287 +++++++++++++++++++------------------
 sysdeps/generic/dl-cache.h |  17 ++-
 2 files changed, 158 insertions(+), 146 deletions(-)

diff --git a/elf/dl-cache.c b/elf/dl-cache.c
index 3aa8ed6c13..02c46ffb0c 100644
--- a/elf/dl-cache.c
+++ b/elf/dl-cache.c
@@ -35,103 +35,141 @@ static struct cache_file *cache;
 static struct cache_file_new *cache_new;
 static size_t cachesize;
 
-/* 1 if cache_data + PTR points into the cache.  */
-#define _dl_cache_verify_ptr(ptr) (ptr < cache_data_size)
-
-#define SEARCH_CACHE(cache) \
-/* We use binary search since the table is sorted in the cache file.	      \
-   The first matching entry in the table is returned.			      \
-   It is important to use the same algorithm as used while generating	      \
-   the cache file.  */							      \
-do									      \
-  {									      \
-    left = 0;								      \
-    right = cache->nlibs - 1;						      \
-									      \
-    while (left <= right)						      \
-      {									      \
-	__typeof__ (cache->libs[0].key) key;				      \
-									      \
-	middle = (left + right) / 2;					      \
-									      \
-	key = cache->libs[middle].key;					      \
-									      \
-	/* Make sure string table indices are not bogus before using	      \
-	   them.  */							      \
-	if (! _dl_cache_verify_ptr (key))				      \
-	  {								      \
-	    cmpres = 1;							      \
-	    break;							      \
-	  }								      \
-									      \
-	/* Actually compare the entry with the key.  */			      \
-	cmpres = _dl_cache_libcmp (name, cache_data + key);		      \
-	if (__glibc_unlikely (cmpres == 0))				      \
-	  {								      \
-	    /* Found it.  LEFT now marks the last entry for which we	      \
-	       know the name is correct.  */				      \
-	    left = middle;						      \
-									      \
-	    /* There might be entries with this name before the one we	      \
-	       found.  So we have to find the beginning.  */		      \
-	    while (middle > 0)						      \
-	      {								      \
-		__typeof__ (cache->libs[0].key) key;			      \
-									      \
-		key = cache->libs[middle - 1].key;			      \
-		/* Make sure string table indices are not bogus before	      \
-		   using them.  */					      \
-		if (! _dl_cache_verify_ptr (key)			      \
-		    /* Actually compare the entry.  */			      \
-		    || _dl_cache_libcmp (name, cache_data + key) != 0)	      \
-		  break;						      \
-		--middle;						      \
-	      }								      \
-									      \
-	    do								      \
-	      {								      \
-		int flags;						      \
-		__typeof__ (cache->libs[0]) *lib = &cache->libs[middle];      \
-									      \
-		/* Only perform the name test if necessary.  */		      \
-		if (middle > left					      \
-		    /* We haven't seen this string so far.  Test whether the  \
-		       index is ok and whether the name matches.  Otherwise   \
-		       we are done.  */					      \
-		    && (! _dl_cache_verify_ptr (lib->key)		      \
-			|| (_dl_cache_libcmp (name, cache_data + lib->key)    \
-			    != 0)))					      \
-		  break;						      \
-									      \
-		flags = lib->flags;					      \
-		if (_dl_cache_check_flags (flags)			      \
-		    && _dl_cache_verify_ptr (lib->value))		      \
-		  {							      \
-		    if (best == NULL || flags == GLRO(dl_correct_cache_id))   \
-		      {							      \
-			HWCAP_CHECK;					      \
-			best = cache_data + lib->value;			      \
-									      \
-			if (flags == GLRO(dl_correct_cache_id))		      \
-			  /* We've found an exact match for the shared	      \
-			     object and no general `ELF' release.  Stop	      \
-			     searching.  */				      \
-			  break;					      \
-		      }							      \
-		  }							      \
-	      }								      \
-	    while (++middle <= right);					      \
-	    break;							      \
-	}								      \
-									      \
-	if (cmpres < 0)							      \
-	  left = middle + 1;						      \
-	else								      \
-	  right = middle - 1;						      \
-      }									      \
-  }									      \
-while (0)
+/* True if PTR is a valid string table index.  */
+static inline bool
+_dl_cache_verify_ptr (uint32_t ptr, size_t string_table_size)
+{
+  return ptr < string_table_size;
+}
+
+/* Compute the address of the element INDEX of the array at LIBS.
+   Conceptually, this is &LIBS[INDEX], but use ENTRY_SIZE for the size
+   of *LIBS.  */
+static inline const struct file_entry *
+_dl_cache_file_entry (const struct file_entry *libs, size_t entry_size,
+		      size_t index)
+{
+  return (const void *) libs + index * entry_size;
+}
+
+/* We use binary search since the table is sorted in the cache file.
+   The first matching entry in the table is returned.  It is important
+   to use the same algorithm as used while generating the cache file.
+   STRING_TABLE_SIZE indicates the maximum offset in STRING_TABLE at
+   which data is mapped; it is not exact.  */
+static const char *
+search_cache (const char *string_table, uint32_t string_table_size,
+	      struct file_entry *libs, uint32_t nlibs, uint32_t entry_size,
+	      const char *name)
+{
+  /* Used by the HWCAP check in the struct file_entry_new case.  */
+  uint64_t platform = _dl_string_platform (GLRO (dl_platform));
+  if (platform != (uint64_t) -1)
+    platform = 1ULL << platform;
+  uint64_t hwcap_mask = GET_HWCAP_MASK ();
+#define _DL_HWCAP_TLS_MASK (1LL << 63)
+  uint64_t hwcap_exclude = ~((GLRO (dl_hwcap) & hwcap_mask)
+			     | _DL_HWCAP_PLATFORM | _DL_HWCAP_TLS_MASK);
+
+  int left = 0;
+  int right = nlibs - 1;
+  const char *best = NULL;
+
+  while (left <= right)
+    {
+      int middle = (left + right) / 2;
+      uint32_t key = _dl_cache_file_entry (libs, entry_size, middle)->key;
+
+      /* Make sure string table indices are not bogus before using
+	 them.  */
+      if (!_dl_cache_verify_ptr (key, string_table_size))
+	return NULL;
 
+      /* Actually compare the entry with the key.  */
+      int cmpres = _dl_cache_libcmp (name, string_table + key);
+      if (__glibc_unlikely (cmpres == 0))
+	{
+	  /* Found it.  LEFT now marks the last entry for which we
+	     know the name is correct.  */
+	  left = middle;
+
+	  /* There might be entries with this name before the one we
+	     found.  So we have to find the beginning.  */
+	  while (middle > 0)
+	    {
+	      key = _dl_cache_file_entry (libs, entry_size, middle - 1)->key;
+	      /* Make sure string table indices are not bogus before
+		 using them.  */
+	      if (!_dl_cache_verify_ptr (key, string_table_size)
+		  /* Actually compare the entry.  */
+		  || _dl_cache_libcmp (name, string_table + key) != 0)
+		break;
+	      --middle;
+	    }
+
+	  do
+	    {
+	      int flags;
+	      const struct file_entry *lib
+		= _dl_cache_file_entry (libs, entry_size, middle);
+
+	      /* Only perform the name test if necessary.  */
+	      if (middle > left
+		  /* We haven't seen this string so far.  Test whether the
+		     index is ok and whether the name matches.  Otherwise
+		     we are done.  */
+		  && (! _dl_cache_verify_ptr (lib->key, string_table_size)
+		      || (_dl_cache_libcmp (name, string_table + lib->key)
+			  != 0)))
+		break;
+
+	      flags = lib->flags;
+	      if (_dl_cache_check_flags (flags)
+		  && _dl_cache_verify_ptr (lib->value, string_table_size))
+		{
+		  if (best == NULL || flags == GLRO (dl_correct_cache_id))
+		    {
+		      if (entry_size >= sizeof (struct file_entry_new))
+			{
+			  /* The entry is large enough to include
+			     HWCAP data.  Check it.  */
+			  struct file_entry_new *libnew
+			    = (struct file_entry_new *) lib;
+
+			  if (libnew->hwcap & hwcap_exclude)
+			    continue;
+			  if (GLRO (dl_osversion)
+			      && libnew->osversion > GLRO (dl_osversion))
+			    continue;
+			  if (_DL_PLATFORMS_COUNT
+			      && (libnew->hwcap & _DL_HWCAP_PLATFORM) != 0
+			      && ((libnew->hwcap & _DL_HWCAP_PLATFORM)
+				  != platform))
+			    continue;
+			}
+
+		      best = string_table + lib->value;
+
+		      if (flags == GLRO (dl_correct_cache_id))
+			/* We've found an exact match for the shared
+			   object and no general `ELF' release.  Stop
+			   searching.  */
+			break;
+		    }
+		}
+	    }
+	  while (++middle <= right);
+	  break;
+	}
+
+      if (cmpres < 0)
+	left = middle + 1;
+      else
+	right = middle - 1;
+    }
+
+  return best;
+}
 
 int
 _dl_cache_libcmp (const char *p1, const char *p2)
@@ -182,12 +220,6 @@ _dl_cache_libcmp (const char *p1, const char *p2)
 char *
 _dl_load_cache_lookup (const char *name)
 {
-  int left, right, middle;
-  int cmpres;
-  const char *cache_data;
-  uint32_t cache_data_size;
-  const char *best;
-
   /* Print a message if the loading of libs is traced.  */
   if (__glibc_unlikely (GLRO(dl_debug_mask) & DL_DEBUG_LIBS))
     _dl_debug_printf (" search cache=%s\n", LD_SO_CACHE);
@@ -265,51 +297,22 @@ _dl_load_cache_lookup (const char *name)
     /* Previously looked for the cache file and didn't find it.  */
     return NULL;
 
-  best = NULL;
-
+  const char *best;
   if (cache_new != (void *) -1)
     {
-      uint64_t platform;
-
-      /* This is where the strings start.  */
-      cache_data = (const char *) cache_new;
-
-      /* Now we can compute how large the string table is.  */
-      cache_data_size = (const char *) cache + cachesize - cache_data;
-
-      platform = _dl_string_platform (GLRO(dl_platform));
-      if (platform != (uint64_t) -1)
-	platform = 1ULL << platform;
-
-      uint64_t hwcap_mask = GET_HWCAP_MASK();
-
-#define _DL_HWCAP_TLS_MASK (1LL << 63)
-      uint64_t hwcap_exclude = ~((GLRO(dl_hwcap) & hwcap_mask)
-				 | _DL_HWCAP_PLATFORM | _DL_HWCAP_TLS_MASK);
-
-      /* Only accept hwcap if it's for the right platform.  */
-#define HWCAP_CHECK \
-      if (lib->hwcap & hwcap_exclude)					      \
-	continue;							      \
-      if (GLRO(dl_osversion) && lib->osversion > GLRO(dl_osversion))	      \
-	continue;							      \
-      if (_DL_PLATFORMS_COUNT						      \
-	  && (lib->hwcap & _DL_HWCAP_PLATFORM) != 0			      \
-	  && (lib->hwcap & _DL_HWCAP_PLATFORM) != platform)		      \
-	continue
-      SEARCH_CACHE (cache_new);
+      const char *string_table = (const char *) cache_new;
+      best = search_cache (string_table, cachesize,
+			   &cache_new->libs[0].entry, cache_new->nlibs,
+			   sizeof (cache_new->libs[0]), name);
     }
   else
     {
-      /* This is where the strings start.  */
-      cache_data = (const char *) &cache->libs[cache->nlibs];
-
-      /* Now we can compute how large the string table is.  */
-      cache_data_size = (const char *) cache + cachesize - cache_data;
-
-#undef HWCAP_CHECK
-#define HWCAP_CHECK do {} while (0)
-      SEARCH_CACHE (cache);
+      const char *string_table = (const char *) &cache->libs[cache->nlibs];
+      uint32_t string_table_size
+	= (const char *) cache + cachesize - string_table;
+      best = search_cache (string_table, string_table_size,
+			   &cache->libs[0], cache->nlibs,
+			   sizeof (cache->libs[0]), name);
     }
 
   /* Print our result if wanted.  */
diff --git a/sysdeps/generic/dl-cache.h b/sysdeps/generic/dl-cache.h
index b154740da9..fec209509d 100644
--- a/sysdeps/generic/dl-cache.h
+++ b/sysdeps/generic/dl-cache.h
@@ -66,8 +66,8 @@
 */
 struct file_entry
 {
-  int flags;		/* This is 1 for an ELF library.  */
-  unsigned int key, value; /* String table indices.  */
+  int32_t flags;		/* This is 1 for an ELF library.  */
+  uint32_t key, value;		/* String table indices.  */
 };
 
 struct cache_file
@@ -84,8 +84,17 @@ struct cache_file
 
 struct file_entry_new
 {
-  int32_t flags;		/* This is 1 for an ELF library.  */
-  uint32_t key, value;		/* String table indices.  */
+  union
+  {
+    /* Fields shared with struct file_entry.  */
+    struct file_entry entry;
+    /* Also expose these fields directly.  */
+    struct
+    {
+      int32_t flags;		/* This is 1 for an ELF library.  */
+      uint32_t key, value;	/* String table indices.  */
+    };
+  };
   uint32_t osversion;		/* Required OS version.	 */
   uint64_t hwcap;		/* Hwcap entry.	 */
 };
-- 
2.25.4

[PATCH 26/30] elf: Implement a string table for ldconfig, with tail merging

[Florian Weimer]

This will be used in ldconfig to reduce the ld.so.cache size slightly.
---
 elf/Makefile           |   2 +-
 elf/stringtable.c      | 201 +++++++++++++++++++++++++++++++++++++++++
 elf/stringtable.h      |  61 +++++++++++++
 elf/stringtable_free.c |  32 +++++++
 elf/tst-stringtable.c  | 140 ++++++++++++++++++++++++++++
 5 files changed, 435 insertions(+), 1 deletion(-)
 create mode 100644 elf/stringtable.c
 create mode 100644 elf/stringtable.h
 create mode 100644 elf/stringtable_free.c
 create mode 100644 elf/tst-stringtable.c

diff --git a/elf/Makefile b/elf/Makefile
index 728cb3b734..06e57300c2 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -170,7 +170,7 @@ tests-container := \
 
 tests := tst-tls9 tst-leaks1 \
 	tst-array1 tst-array2 tst-array3 tst-array4 tst-array5 \
-	tst-auxv
+	tst-auxv tst-stringtable
 tests-internal := tst-tls1 tst-tls2 $(tests-static-internal)
 tests-static := $(tests-static-normal) $(tests-static-internal)
 
diff --git a/elf/stringtable.c b/elf/stringtable.c
new file mode 100644
index 0000000000..f9ade50249
--- /dev/null
+++ b/elf/stringtable.c
@@ -0,0 +1,201 @@
+/* String tables for ld.so.cache construction.  Implementation.
+   This file is part of the GNU C Library.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published
+   by the Free Software Foundation; version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see <https://www.gnu.org/licenses/>.  */
+
+#include <assert.h>
+#include <error.h>
+#include <ldconfig.h>
+#include <libintl.h>
+#include <stdlib.h>
+#include <string.h>
+#include <stringtable.h>
+
+static void
+stringtable_init (struct stringtable *table)
+{
+  table->count = 0;
+  table->allocated = 16;
+  table->entries = xcalloc (table->allocated, sizeof (table->entries[0]));
+}
+
+/* 32-bit FNV-1a hash function.  */
+static uint32_t
+fnv1a (const char *string, size_t length)
+{
+  const unsigned char *p = (const unsigned char *) string;
+  uint32_t hash = 2166136261U;
+  for (size_t i = 0; i < length; ++i)
+    {
+      hash ^= p[i];
+      hash *= 16777619U;
+    }
+  return hash;
+}
+
+/* Double the capacity of the hash table.  */
+static void
+stringtable_rehash (struct stringtable *table)
+{
+  /* Cannot overflow because the old allocation size (in bytes) is
+     larger.  */
+  uint32_t new_allocated = table->allocated * 2;
+  struct stringtable_entry **new_entries
+    = xcalloc (new_allocated, sizeof (table->entries[0]));
+
+  uint32_t mask = new_allocated - 1;
+  for (uint32_t i = 0; i < table->allocated; ++i)
+    for (struct stringtable_entry *e = table->entries[i]; e != NULL; )
+      {
+        struct stringtable_entry *next = e->next;
+        uint32_t hash = fnv1a (e->string, e->length);
+        uint32_t new_index = hash & mask;
+        e->next = new_entries[new_index];
+        new_entries[new_index] = e;
+        e = next;
+      }
+
+  free (table->entries);
+  table->entries = new_entries;
+  table->allocated = new_allocated;
+}
+
+struct stringtable_entry *
+stringtable_intern (struct stringtable *table, const char *string)
+{
+  if (table->allocated == 0)
+    stringtable_init (table);
+
+  size_t length = strlen (string);
+  if (length > (1U << 30))
+    error (EXIT_FAILURE, 0, _("String table string is too long"));
+  uint32_t hash = fnv1a (string, length);
+
+  /* Return a previously-existing entry.  */
+  for (struct stringtable_entry *e
+         = table->entries[hash & (table->allocated - 1)];
+       e != NULL; e = e->next)
+    if (e->length == length && memcmp (e->string, string, length) == 0)
+      return e;
+
+  /* Increase the size of the table if necessary.  Keep utilization
+     below two thirds.  */
+  if (table->count >= (1U << 30))
+    error (EXIT_FAILURE, 0, _("String table has too many entries"));
+  if (table->count * 3 > table->allocated * 2)
+    stringtable_rehash (table);
+
+  /* Add the new table entry.  */
+  ++table->count;
+  struct stringtable_entry *e
+    = xmalloc (offsetof (struct stringtable_entry, string) + length + 1);
+  uint32_t index = hash & (table->allocated - 1);
+  e->next = table->entries[index];
+  table->entries[index] = e;
+  e->length = length;
+  e->offset = 0;
+  memcpy (e->string, string, length + 1);
+  return e;
+}
+
+/* Sort reversed strings in lexicographic order.  This is used for tail
+   merging.  */
+static int
+finalize_compare (const void *l, const void *r)
+{
+  struct stringtable_entry *left = *(struct stringtable_entry **) l;
+  struct stringtable_entry *right = *(struct stringtable_entry **) r;
+  size_t to_compare;
+  if (left->length < right->length)
+    to_compare = left->length;
+  else
+    to_compare = right->length;
+  for (ssize_t i = to_compare - 1; i >= 0; --i)
+    {
+      unsigned char lch = left->string[i];
+      unsigned char rch = right->string[i];
+      if (lch != rch)
+        return lch - rch;
+    }
+  if (left->length == right->length)
+    return 0;
+  else if (left->length < right->length)
+    /* Longer strings should come first.  */
+    return 1;
+  else
+    return -1;
+}
+
+void
+stringtable_finalize (struct stringtable *table,
+                      struct stringtable_finalized *result)
+{
+  if (table->count == 0)
+    {
+      result->strings = xstrdup ("");
+      result->size = 0;
+      return;
+    }
+
+  /* Optimize the order of the strings.  */
+  struct stringtable_entry **array = xcalloc (table->count, sizeof (*array));
+  {
+    size_t j = 0;
+    for (uint32_t i = 0; i < table->allocated; ++i)
+      for (struct stringtable_entry *e = table->entries[i]; e != NULL;
+           e = e->next)
+        {
+          array[j] = e;
+          ++j;
+        }
+    assert (j == table->count);
+  }
+  qsort (array, table->count, sizeof (*array), finalize_compare);
+
+  /* Assign offsets, using table sharing if possible.  */
+  array[0]->offset = 0;
+  for (uint32_t j = 1; j < table->count; ++j)
+    {
+      struct stringtable_entry *previous = array[j - 1];
+      struct stringtable_entry *current = array[j];
+      if (previous->length >= current->length
+          && memcmp (&previous->string[previous->length - current->length],
+                     current->string, current->length) == 0)
+        current->offset = (previous->offset + previous->length
+                           - current->length);
+      else if (__builtin_add_overflow (previous->offset,
+                                       previous->length + 1,
+                                       &current->offset))
+        error (EXIT_FAILURE, 0, _("String table is too large"));
+    }
+
+  /* Allocate the result string.  */
+  {
+    struct stringtable_entry *last = array[table->count - 1];
+    if (__builtin_add_overflow (last->offset, last->length + 1,
+                                &result->size))
+      error (EXIT_FAILURE, 0, _("String table is too large"));
+  }
+  /* The strings are copied from the hash table, so the array is no
+     longer needed.  */
+  free (array);
+  result->strings = xcalloc (result->size, 1);
+
+  /* Copy the strings.  */
+  for (uint32_t i = 0; i < table->allocated; ++i)
+    for (struct stringtable_entry *e = table->entries[i]; e != NULL;
+         e = e->next)
+      if (result->strings[e->offset] == '\0')
+        memcpy (&result->strings[e->offset], e->string, e->length + 1);
+}
diff --git a/elf/stringtable.h b/elf/stringtable.h
new file mode 100644
index 0000000000..e35b6c67fd
--- /dev/null
+++ b/elf/stringtable.h
@@ -0,0 +1,61 @@
+/* String tables for ld.so.cache construction.
+   This file is part of the GNU C Library.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published
+   by the Free Software Foundation; version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see <https://www.gnu.org/licenses/>.  */
+
+#ifndef _STRINGTABLE_H
+#define _STRINGTABLE_H
+
+#include <stddef.h>
+#include <stdint.h>
+
+/* An entry in the string table.  Only the length and string fields are
+   expected to be used outside the string table code.  */
+struct stringtable_entry
+{
+  struct stringtable_entry *next; /* For collision resolution.  */
+  uint32_t length;                /* Length of then string.  */
+  uint32_t offset;                /* From start of finalized table.  */
+  char string[];                  /* Null-terminated string.  */
+};
+
+/* A string table.  Zero-initialization produces a valid atable.  */
+struct stringtable
+{
+  struct stringtable_entry **entries;
+  uint32_t count;                 /* Number of elements in the table.  */
+  uint32_t allocated;             /* Length of the entries array.  */
+};
+
+/* Adds STRING to TABLE.  May return the address of an existing entry.  */
+struct stringtable_entry *stringtable_intern (struct stringtable *table,
+                                              const char *string);
+
+/* Result of stringtable_finalize.  SIZE bytes at STRINGS should be
+   written to the file.  */
+struct stringtable_finalized
+{
+  char *strings;
+  size_t size;
+};
+
+/* Assigns offsets to string table entries and computes the serialized
+   form of the string table.  */
+void stringtable_finalize (struct stringtable *table,
+                           struct stringtable_finalized *result);
+
+/* Deallocate the string table (but not the TABLE pointer itself).  */
+void stringtable_free (struct stringtable *table);
+
+#endif /* _STRINGTABLE_H */
diff --git a/elf/stringtable_free.c b/elf/stringtable_free.c
new file mode 100644
index 0000000000..0e5296e429
--- /dev/null
+++ b/elf/stringtable_free.c
@@ -0,0 +1,32 @@
+/* String tables for ld.so.cache construction.  Deallocation (for tests only).
+   This file is part of the GNU C Library.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published
+   by the Free Software Foundation; version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see <https://www.gnu.org/licenses/>.  */
+
+#include <stdlib.h>
+#include <stringtable.h>
+
+void
+stringtable_free (struct stringtable *table)
+{
+  for (uint32_t i = 0; i < table->allocated; ++i)
+    for (struct stringtable_entry *e = table->entries[i]; e != NULL; )
+      {
+        struct stringtable_entry *next = e->next;
+        free (e);
+        e = next;
+      }
+  free (table->entries);
+  *table = (struct stringtable) { 0, };
+}
diff --git a/elf/tst-stringtable.c b/elf/tst-stringtable.c
new file mode 100644
index 0000000000..78ca5434df
--- /dev/null
+++ b/elf/tst-stringtable.c
@@ -0,0 +1,140 @@
+/* Unit test for ldconfig string tables.
+   This file is part of the GNU C Library.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published
+   by the Free Software Foundation; version 2 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program; if not, see <https://www.gnu.org/licenses/>.  */
+
+#include <stdlib.h>
+#include <string.h>
+#include <stringtable.h>
+#include <support/check.h>
+#include <support/support.h>
+
+static int
+do_test (void)
+{
+  /* Empty string table.  */
+  {
+    struct stringtable s = { 0, };
+    struct stringtable_finalized f;
+    stringtable_finalize (&s, &f);
+    TEST_COMPARE_STRING (f.strings, "");
+    TEST_COMPARE (f.size, 0);
+    free (f.strings);
+    stringtable_free (&s);
+  }
+
+  /* String table with one empty string.  */
+  {
+    struct stringtable s = { 0, };
+    struct stringtable_entry *e = stringtable_intern (&s, "");
+    TEST_COMPARE_STRING (e->string, "");
+    TEST_COMPARE (e->length, 0);
+    TEST_COMPARE (s.count, 1);
+
+    struct stringtable_finalized f;
+    stringtable_finalize (&s, &f);
+    TEST_COMPARE (e->offset, 0);
+    TEST_COMPARE_STRING (f.strings, "");
+    TEST_COMPARE (f.size, 1);
+    free (f.strings);
+    stringtable_free (&s);
+  }
+
+  /* String table with one non-empty string.  */
+  {
+    struct stringtable s = { 0, };
+    struct stringtable_entry *e = stringtable_intern (&s, "name");
+    TEST_COMPARE_STRING (e->string, "name");
+    TEST_COMPARE (e->length, 4);
+    TEST_COMPARE (s.count, 1);
+
+    struct stringtable_finalized f;
+    stringtable_finalize (&s, &f);
+    TEST_COMPARE (e->offset, 0);
+    TEST_COMPARE_STRING (f.strings, "name");
+    TEST_COMPARE (f.size, 5);
+    free (f.strings);
+    stringtable_free (&s);
+  }
+
+  /* Two strings, one is a prefix of the other.  Tail-merging can only
+     happen in one way in this case.  */
+  {
+    struct stringtable s = { 0, };
+    struct stringtable_entry *suffix = stringtable_intern (&s, "suffix");
+    TEST_COMPARE_STRING (suffix->string, "suffix");
+    TEST_COMPARE (suffix->length, 6);
+    TEST_COMPARE (s.count, 1);
+
+    struct stringtable_entry *prefix
+      = stringtable_intern (&s, "prefix-suffix");
+    TEST_COMPARE_STRING (prefix->string, "prefix-suffix");
+    TEST_COMPARE (prefix->length, strlen ("prefix-suffix"));
+    TEST_COMPARE (s.count, 2);
+
+    struct stringtable_finalized f;
+    stringtable_finalize (&s, &f);
+    TEST_COMPARE (prefix->offset, 0);
+    TEST_COMPARE (suffix->offset, strlen ("prefix-"));
+    TEST_COMPARE_STRING (f.strings, "prefix-suffix");
+    TEST_COMPARE (f.size, sizeof ("prefix-suffix"));
+    free (f.strings);
+    stringtable_free (&s);
+  }
+
+  /* String table with various shared prefixes.  Triggers hash
+     resizing.  */
+  {
+    enum { count = 1500 };
+    char *strings[2 * count];
+    struct stringtable_entry *entries[2 * count];
+    struct stringtable s = { 0, };
+    for (int i = 0; i < count; ++i)
+      {
+        strings[i] = xasprintf ("%d", i);
+        entries[i] = stringtable_intern (&s, strings[i]);
+        TEST_COMPARE (entries[i]->length, strlen (strings[i]));
+        TEST_COMPARE_STRING (entries[i]->string, strings[i]);
+        strings[i + count] = xasprintf ("prefix/%d", i);
+        entries[i + count] = stringtable_intern (&s, strings[i + count]);
+        TEST_COMPARE (entries[i + count]->length, strlen (strings[i + count]));
+        TEST_COMPARE_STRING (entries[i + count]->string, strings[i + count]);
+      }
+
+    struct stringtable_finalized f;
+    stringtable_finalize (&s, &f);
+
+    for (int i = 0; i < 2 * count; ++i)
+      {
+        TEST_COMPARE (entries[i]->length, strlen (strings[i]));
+        TEST_COMPARE_STRING (entries[i]->string, strings[i]);
+        TEST_COMPARE_STRING (f.strings + entries[i]->offset, strings[i]);
+        free (strings[i]);
+      }
+
+    free (f.strings);
+    stringtable_free (&s);
+  }
+
+  return 0;
+}
+
+#include <support/test-driver.c>
+
+/* Re-compile the string table implementation here.  It is not
+   possible to link against the actual build because it was built for
+   use in ldconfig.  */
+#define _(arg) arg
+#include "stringtable.c"
+#include "stringtable_free.c"
-- 
2.25.4

[PATCH 27/30] elf: Implement tail merging of strings in ldconfig

[Florian Weimer]

This simplifies the string table construction in elf/cache.c
because there is no more need to keep track of offsets explicitly;
the string table implementation does this internally.

This change slightly reduces the size of the cache on disk.  The
file format does not change as a result.  The strings are
null-terminated, without explicit length, so tail merging is
transparent to readers.
---
 elf/Makefile |  3 +-
 elf/cache.c  | 84 ++++++++++++++++++++++++++++------------------------
 2 files changed, 48 insertions(+), 39 deletions(-)

diff --git a/elf/Makefile b/elf/Makefile
index 06e57300c2..987ef5484b 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -118,7 +118,8 @@ others-static	+= ldconfig
 others		+= ldconfig
 install-rootsbin += ldconfig
 
-ldconfig-modules := cache readlib xmalloc xstrdup chroot_canon static-stubs
+ldconfig-modules := cache readlib xmalloc xstrdup chroot_canon static-stubs \
+  stringtable
 extra-objs	+= $(ldconfig-modules:=.o)
 others-extras   = $(ldconfig-modules)
 endif
diff --git a/elf/cache.c b/elf/cache.c
index ee4d74fed1..4b2df86dc5 100644
--- a/elf/cache.c
+++ b/elf/cache.c
@@ -35,11 +35,15 @@
 #include <ldconfig.h>
 #include <dl-cache.h>
 #include <version.h>
+#include <stringtable.h>
+
+/* Used to store library names, paths, and other strings.  */
+struct stringtable strings;
 
 struct cache_entry
 {
-  char *lib;			/* Library name.  */
-  char *path;			/* Path to find library.  */
+  struct stringtable_entry *lib; /* Library name.  */
+  struct stringtable_entry *path; /* Path to find library.  */
   int flags;			/* Flags to indicate kind of library.  */
   unsigned int osversion;	/* Required OS version.  */
   uint64_t hwcap;		/* Important hardware capabilities.  */
@@ -300,7 +304,7 @@ static int
 compare (const struct cache_entry *e1, const struct cache_entry *e2)
 {
   /* We need to swap entries here to get the correct sort order.  */
-  int res = _dl_cache_libcmp (e2->lib, e1->lib);
+  int res = _dl_cache_libcmp (e2->lib->string, e1->lib->string);
   if (res == 0)
     {
       if (e1->flags < e2->flags)
@@ -369,26 +373,24 @@ save_cache (const char *cache_name)
 {
   /* The cache entries are sorted already, save them in this order. */
 
-  /* Count the length of all strings.  */
-  /* The old format doesn't contain hwcap entries and doesn't contain
-     libraries in subdirectories with hwcaps entries.  Count therefore
-     also all entries with hwcap == 0.  */
-  size_t total_strlen = 0;
   struct cache_entry *entry;
   /* Number of cache entries.  */
   int cache_entry_count = 0;
-  /* Number of normal cache entries.  */
+  /* The old format doesn't contain hwcap entries and doesn't contain
+     libraries in subdirectories with hwcaps entries.  Count therefore
+     also all entries with hwcap == 0.  */
   int cache_entry_old_count = 0;
 
   for (entry = entries; entry != NULL; entry = entry->next)
     {
-      /* Account the final NULs.  */
-      total_strlen += strlen (entry->lib) + strlen (entry->path) + 2;
       ++cache_entry_count;
       if (entry->hwcap == 0)
 	++cache_entry_old_count;
     }
 
+  struct stringtable_finalized strings_finalized;
+  stringtable_finalize (&strings, &strings_finalized);
+
   /* Create the on disk cache structure.  */
   struct cache_file *file_entries = NULL;
   size_t file_entries_size = 0;
@@ -432,7 +434,7 @@ save_cache (const char *cache_name)
 	      sizeof CACHE_VERSION - 1);
 
       file_entries_new->nlibs = cache_entry_count;
-      file_entries_new->len_strings = total_strlen;
+      file_entries_new->len_strings = strings_finalized.size;
       file_entries_new->flags = cache_file_new_flags_endian;
     }
 
@@ -449,20 +451,20 @@ save_cache (const char *cache_name)
     str_offset = 0;
 
   /* An array for all strings.  */
-  char *strings = xmalloc (total_strlen);
-  char *str = strings;
   int idx_old;
   int idx_new;
 
   for (idx_old = 0, idx_new = 0, entry = entries; entry != NULL;
        entry = entry->next, ++idx_new)
     {
-      /* First the library.  */
       if (opt_format != 2 && entry->hwcap == 0)
 	{
 	  file_entries->libs[idx_old].flags = entry->flags;
 	  /* XXX: Actually we can optimize here and remove duplicates.  */
 	  file_entries->libs[idx_old].key = str_offset + pad;
+	  file_entries->libs[idx_new].key = str_offset + entry->lib->offset;
+	  file_entries->libs[idx_new].value
+	    = str_offset + entry->path->offset;
 	}
       if (opt_format != 0)
 	{
@@ -473,20 +475,12 @@ save_cache (const char *cache_name)
 	  file_entries_new->libs[idx_new].flags = entry->flags;
 	  file_entries_new->libs[idx_new].osversion = entry->osversion;
 	  file_entries_new->libs[idx_new].hwcap = entry->hwcap;
-	  file_entries_new->libs[idx_new].key = str_offset;
+	  file_entries_new->libs[idx_new].key
+	    = str_offset + entry->lib->offset;
+	  file_entries_new->libs[idx_new].value
+	    = str_offset + entry->path->offset;
 	}
 
-      size_t len = strlen (entry->lib) + 1;
-      str = mempcpy (str, entry->lib, len);
-      str_offset += len;
-      /* Then the path.  */
-      if (opt_format != 2 && entry->hwcap == 0)
-	file_entries->libs[idx_old].value = str_offset + pad;
-      if (opt_format != 0)
-	file_entries_new->libs[idx_new].value = str_offset;
-      len = strlen (entry->path) + 1;
-      str = mempcpy (str, entry->path, len);
-      str_offset += len;
       /* Ignore entries with hwcap for old format.  */
       if (entry->hwcap == 0)
 	++idx_old;
@@ -511,7 +505,7 @@ save_cache (const char *cache_name)
 	extension_offset += pad;
       extension_offset += file_entries_new_size;
     }
-  extension_offset += total_strlen;
+  extension_offset += strings_finalized.size;
   extension_offset = roundup (extension_offset, 4); /* Provide alignment.  */
   if (opt_format != 0)
     file_entries_new->extension_offset = extension_offset;
@@ -551,7 +545,8 @@ save_cache (const char *cache_name)
 	error (EXIT_FAILURE, errno, _("Writing of cache data failed"));
     }
 
-  if (write (fd, strings, total_strlen) != (ssize_t) total_strlen)
+  if (write (fd, strings_finalized.strings, strings_finalized.size)
+      != (ssize_t) strings_finalized.size)
     error (EXIT_FAILURE, errno, _("Writing of cache data failed"));
 
   if (opt_format != 0)
@@ -580,7 +575,7 @@ save_cache (const char *cache_name)
   /* Free all allocated memory.  */
   free (file_entries_new);
   free (file_entries);
-  free (strings);
+  free (strings_finalized.strings);
 
   while (entries)
     {
@@ -596,14 +591,27 @@ void
 add_to_cache (const char *path, const char *lib, int flags,
 	      unsigned int osversion, uint64_t hwcap)
 {
-  size_t liblen = strlen (lib) + 1;
-  size_t len = liblen + strlen (path) + 1;
-  struct cache_entry *new_entry
-    = xmalloc (sizeof (struct cache_entry) + liblen + len);
-
-  new_entry->lib = memcpy ((char *) (new_entry + 1), lib, liblen);
-  new_entry->path = new_entry->lib + liblen;
-  snprintf (new_entry->path, len, "%s/%s", path, lib);
+  struct cache_entry *new_entry = xmalloc (sizeof (*new_entry));
+
+  struct stringtable_entry *path_interned;
+  {
+    /* Use a small, on-stack buffer in most cases.  */
+    char buf[200];
+    int ret = snprintf (buf, sizeof (buf), "%s/%s", path, lib);
+    if (ret < 0 || ret >= sizeof (buf) - 1)
+      {
+	char *p;
+	if (asprintf (&p, "%s/%s", path, lib) < 0)
+	  error (EXIT_FAILURE, errno, _("Could not create library path"));
+	path_interned = stringtable_intern (&strings, p);
+	free (p);
+      }
+    else
+      path_interned = stringtable_intern (&strings, buf);
+  }
+
+  new_entry->lib = stringtable_intern (&strings, lib);
+  new_entry->path = path_interned;
   new_entry->flags = flags;
   new_entry->osversion = osversion;
   new_entry->hwcap = hwcap;
-- 
2.25.4

[PATCH 28/30] elf: In ldconfig, extract the new_sub_entry function from search_dir

[Florian Weimer]

---
 elf/ldconfig.c | 34 +++++++++++++++++++++-------------
 1 file changed, 21 insertions(+), 13 deletions(-)

diff --git a/elf/ldconfig.c b/elf/ldconfig.c
index 0c090dca15..3768267bac 100644
--- a/elf/ldconfig.c
+++ b/elf/ldconfig.c
@@ -328,6 +328,23 @@ warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\n\
 	   "Andreas Jaeger");
 }
 
+/* Allocate a new subdirectory with full path PATH under ENTRY, using
+   inode data from *ST.  */
+static struct dir_entry *
+new_sub_entry (const struct dir_entry *entry, const char *path,
+	       const struct stat64 *st)
+{
+  struct dir_entry *new_entry = xmalloc (sizeof (struct dir_entry));
+  new_entry->from_file = entry->from_file;
+  new_entry->from_line = entry->from_line;
+  new_entry->path = xstrdup (path);
+  new_entry->flag = entry->flag;
+  new_entry->next = NULL;
+  new_entry->ino = st->st_ino;
+  new_entry->dev = st->st_dev;
+  return new_entry;
+}
+
 /* Add a single directory entry.  */
 static void
 add_single_dir (struct dir_entry *entry, int verbose)
@@ -823,26 +840,17 @@ search_dir (const struct dir_entry *entry)
 
       if (is_dir && is_hwcap_platform (direntry->d_name))
 	{
-	  /* Handle subdirectory later.  */
-	  struct dir_entry *new_entry;
-
-	  new_entry = xmalloc (sizeof (struct dir_entry));
-	  new_entry->from_file = entry->from_file;
-	  new_entry->from_line = entry->from_line;
-	  new_entry->path = xstrdup (file_name);
-	  new_entry->flag = entry->flag;
-	  new_entry->next = NULL;
 	  if (!is_link
 	      && direntry->d_type != DT_UNKNOWN
 	      && __builtin_expect (lstat64 (real_file_name, &lstat_buf), 0))
 	    {
 	      error (0, errno, _("Cannot lstat %s"), file_name);
-	      free (new_entry->path);
-	      free (new_entry);
 	      continue;
 	    }
-	  new_entry->ino = lstat_buf.st_ino;
-	  new_entry->dev = lstat_buf.st_dev;
+
+	  /* Handle subdirectory later.  */
+	  struct dir_entry *new_entry = new_sub_entry (entry, file_name,
+						       &lstat_buf);
 	  add_single_dir (new_entry, 0);
 	  continue;
 	}
-- 
2.25.4

[PATCH 29/30] elf: Process glibc-hwcaps subdirectories in ldconfig

[Florian Weimer]

Libraries from these subdirectories are added to the cache
with a special hwcap bit DL_CACHE_HWCAP_EXTENSION, so that
they are ignored by older dynamic loaders.
---
 elf/cache.c                | 258 ++++++++++++++++++++++++++++++++-----
 elf/ldconfig.c             | 153 +++++++++++++++++++---
 sysdeps/generic/dl-cache.h |  51 +++++++-
 sysdeps/generic/ldconfig.h |  18 ++-
 4 files changed, 426 insertions(+), 54 deletions(-)

diff --git a/elf/cache.c b/elf/cache.c
index 4b2df86dc5..a61f5ca50a 100644
--- a/elf/cache.c
+++ b/elf/cache.c
@@ -40,6 +40,105 @@
 /* Used to store library names, paths, and other strings.  */
 struct stringtable strings;
 
+/* Keeping track of "glibc-hwcaps" subdirectories.  During cache
+   construction, a linear search by name is performed to deduplicate
+   entries.  */
+struct glibc_hwcaps_subdirectory
+{
+  struct glibc_hwcaps_subdirectory *next;
+
+  /* Interned string with the subdirectory name.  */
+  struct stringtable_entry *name;
+
+  /* Array index in the cache_extension_tag_glibc_hwcaps section in
+     the stored cached file.  This is computed after all the
+     subdirectories have been processed, so that subdirectory names in
+     the extension section can be sorted.  */
+  uint32_t section_index;
+
+  /* True if the subdirectory is actually used for anything.  */
+  bool used;
+};
+
+const char *
+glibc_hwcaps_subdirectory_name (struct glibc_hwcaps_subdirectory *dir)
+{
+  return dir->name->string;
+}
+
+/* Linked list of known hwcaps subdirecty names.  */
+static struct glibc_hwcaps_subdirectory *hwcaps;
+
+struct glibc_hwcaps_subdirectory *
+new_glibc_hwcaps_subdirectory (const char *name)
+{
+  struct stringtable_entry *name_interned
+    = stringtable_intern (&strings, name);
+  for (struct glibc_hwcaps_subdirectory *p = hwcaps; p != NULL; p = p->next)
+    if (p->name == name_interned)
+      return p;
+  struct glibc_hwcaps_subdirectory *p = xmalloc (sizeof (p));
+  p->next = hwcaps;
+  p->name = name_interned;
+  p->section_index = 0;
+  p->used = false;
+  hwcaps = p;
+  return p;
+}
+
+/* Helper for sorting struct glibc_hwcaps_subdirectory elements by
+   name.  */
+static int
+assign_glibc_hwcaps_indices_compare (const void *l, const void *r)
+{
+  const struct glibc_hwcaps_subdirectory *left
+    = *(struct glibc_hwcaps_subdirectory **)l;
+  const struct glibc_hwcaps_subdirectory *right
+    = *(struct glibc_hwcaps_subdirectory **)r;
+  return strcmp (left->name->string, right->name->string);
+}
+
+/* Count the number of hwcaps subdirectories which are actually
+   used.  */
+static size_t
+glibc_hwcaps_count (void)
+{
+  size_t count = 0;
+  for (struct glibc_hwcaps_subdirectory *p = hwcaps; p != NULL; p = p->next)
+    if (p->used)
+      ++count;
+  return count;
+}
+
+/* Compute the section_index fields for all   */
+static void
+assign_glibc_hwcaps_indices (void)
+{
+  /* Convert the linked list into an array, so that we can use qsort.
+     Only copy the subdirectories which are actually used.  */
+  size_t count = glibc_hwcaps_count ();
+  struct glibc_hwcaps_subdirectory **array
+    = xmalloc (sizeof (*array) * count);
+  {
+    size_t i = 0;
+    for (struct glibc_hwcaps_subdirectory *p = hwcaps; p != NULL; p = p->next)
+      if (p->used)
+	{
+	  array[i] = p;
+	  ++i;
+	}
+    assert (i == count);
+  }
+
+  qsort (array, count, sizeof (*array), assign_glibc_hwcaps_indices_compare);
+
+  /* Assign the array indices.  */
+  for (size_t i = 0; i < count; ++i)
+    array[i]->section_index = i;
+
+  free (array);
+}
+
 struct cache_entry
 {
   struct stringtable_entry *lib; /* Library name.  */
@@ -48,6 +147,10 @@ struct cache_entry
   unsigned int osversion;	/* Required OS version.  */
   uint64_t hwcap;		/* Important hardware capabilities.  */
   int bits_hwcap;		/* Number of bits set in hwcap.  */
+
+  /* glibc-hwcaps subdirectory.  If not NULL, hwcap must be zero.  */
+  struct glibc_hwcaps_subdirectory *hwcaps;
+
   struct cache_entry *next;	/* Next entry in list.  */
 };
 
@@ -60,7 +163,7 @@ static const char *flag_descr[] =
 /* Print a single entry.  */
 static void
 print_entry (const char *lib, int flag, unsigned int osversion,
-	     uint64_t hwcap, const char *key)
+	     uint64_t hwcap, const char *hwcap_string, const char *key)
 {
   printf ("\t%s (", lib);
   switch (flag & FLAG_TYPE_MASK)
@@ -132,7 +235,9 @@ print_entry (const char *lib, int flag, unsigned int osversion,
       printf (",%d", flag & FLAG_REQUIRED_MASK);
       break;
     }
-  if (hwcap != 0)
+  if (hwcap_string != NULL)
+    printf (", hwcap: \"%s\"", hwcap_string);
+  else if (hwcap != 0)
     printf (", hwcap: %#.16" PRIx64, hwcap);
   if (osversion != 0)
     {
@@ -158,6 +263,29 @@ print_entry (const char *lib, int flag, unsigned int osversion,
   printf (") => %s\n", key);
 }
 
+/* Returns the string with the name of the glibcs-hwcaps subdirectory
+   associated with ENTRY->hwcap.  file_base must be the base address
+   for string table indices.  */
+static const char *
+glibc_hwcaps_string (struct cache_extension_all_loaded *ext,
+		     const void *file_base, size_t file_size,
+		     struct file_entry_new *entry)
+{
+  const uint32_t *hwcaps_array
+    = ext->sections[cache_extension_tag_glibc_hwcaps].base;
+  if (dl_cache_hwcap_extension (entry) && hwcaps_array != NULL)
+    {
+      uint32_t index = (uint32_t) entry->hwcap;
+      if (index < ext->sections[cache_extension_tag_glibc_hwcaps].size / 4)
+	{
+	  uint32_t string_table_index = hwcaps_array[index];
+	  if (string_table_index < file_size)
+	    return file_base + string_table_index;
+	}
+    }
+  return NULL;
+}
+
 /* Print an error and exit if the new-file cache is internally
    inconsistent.  */
 static void
@@ -167,9 +295,7 @@ check_new_cache (struct cache_file_new *cache)
     error (EXIT_FAILURE, 0, _("Cache file has wrong endianness.\n"));
 }
 
-/* Print the extension information at the cache at start address
-   FILE_BASE, of ltength FILE_SIZE bytes.  The new-format cache header
-   is at CACHE, and the file name for diagnostics is CACHE_NAME.  */
+/* Print the extension information in *EXT.  */
 static void
 print_extensions (struct cache_extension_all_loaded *ext)
 {
@@ -266,7 +392,7 @@ print_cache (const char *cache_name)
       /* Print everything.  */
       for (unsigned int i = 0; i < cache->nlibs; i++)
 	print_entry (cache_data + cache->libs[i].key,
-		     cache->libs[i].flags, 0, 0,
+		     cache->libs[i].flags, 0, 0, NULL,
 		     cache_data + cache->libs[i].value);
     }
   else if (format == 1)
@@ -281,11 +407,16 @@ print_cache (const char *cache_name)
 
       /* Print everything.  */
       for (unsigned int i = 0; i < cache_new->nlibs; i++)
-	print_entry (cache_data + cache_new->libs[i].key,
-		     cache_new->libs[i].flags,
-		     cache_new->libs[i].osversion,
-		     cache_new->libs[i].hwcap,
-		     cache_data + cache_new->libs[i].value);
+	{
+	  const char *hwcaps_string
+	    = glibc_hwcaps_string (&ext, cache, cache_size,
+				   &cache_new->libs[i]);
+	  print_entry (cache_data + cache_new->libs[i].key,
+		       cache_new->libs[i].flags,
+		       cache_new->libs[i].osversion,
+		       cache_new->libs[i].hwcap, hwcaps_string,
+		       cache_data + cache_new->libs[i].value);
+	}
       print_extensions (&ext);
     }
   /* Cleanup.  */
@@ -311,8 +442,22 @@ compare (const struct cache_entry *e1, const struct cache_entry *e2)
 	return 1;
       else if (e1->flags > e2->flags)
 	return -1;
+      /* Keep the glibc-hwcaps extension entries before the regular
+	 entries, and sort them by their names.  search_cache in
+	 dl-cache.c stops searching once the first non-extension entry
+	 is found, so the extension entries need to come first.  */
+      else if (e1->hwcaps != NULL && e2->hwcaps == NULL)
+	return -1;
+      else if (e1->hwcaps == NULL && e2->hwcaps != NULL)
+	return 1;
+      else if (e1->hwcaps != NULL && e2->hwcaps != NULL)
+	{
+	  res = strcmp (e1->hwcaps->name->string, e2->hwcaps->name->string);
+	  if (res != 0)
+	    return res;
+	}
       /* Sort by most specific hwcap.  */
-      else if (e2->bits_hwcap > e1->bits_hwcap)
+      if (e2->bits_hwcap > e1->bits_hwcap)
 	return 1;
       else if (e2->bits_hwcap < e1->bits_hwcap)
 	return -1;
@@ -337,30 +482,65 @@ enum
 			      * sizeof (struct cache_extension_section)))
   };
 
-/* Write the cache extensions to FD.  The extension directory is
-   assumed to be located at CACHE_EXTENSION_OFFSET.  */
+/* Write the cache extensions to FD.  The string table is shifted by
+   STRING_TABLE_OFFSET.  The extension directory is assumed to be
+   located at CACHE_EXTENSION_OFFSET.  assign_glibc_hwcaps_indices
+   must have been called.  */
 static void
-write_extensions (int fd, uint32_t cache_extension_offset)
+write_extensions (int fd, uint32_t str_offset,
+		  uint32_t cache_extension_offset)
 {
   assert ((cache_extension_offset % 4) == 0);
 
+  /* The length and contents of the glibc-hwcaps section.  */
+  uint32_t hwcaps_count = glibc_hwcaps_count ();
+  uint32_t hwcaps_offset = cache_extension_offset + cache_extension_size;
+  uint32_t hwcaps_size = hwcaps_count * sizeof (uint32_t);
+  uint32_t *hwcaps_array = xmalloc (hwcaps_size);
+  for (struct glibc_hwcaps_subdirectory *p = hwcaps; p != NULL; p = p->next)
+    if (p->used)
+      hwcaps_array[p->section_index] = str_offset + p->name->offset;
+
+  /* This is the offset of the generator string.  */
+  uint32_t generator_offset = hwcaps_offset;
+  if (hwcaps_count == 0)
+    /* There is no section for the hwcaps subdirectories.  */
+    generator_offset -= sizeof (struct cache_extension_section);
+  else
+    /* The string table indices for the hwcaps subdirectories shift
+       the generator string backwards.  */
+    generator_offset += hwcaps_count * sizeof (uint32_t);
+
   struct cache_extension *ext = xmalloc (cache_extension_size);
   ext->magic = cache_extension_magic;
-  ext->count = cache_extension_count;
 
-  for (int i = 0; i < cache_extension_count; ++i)
-    {
-      ext->sections[i].tag = i;
-      ext->sections[i].flags = 0;
-    }
+  /* Extension index current being filled.  */
+  size_t xid = 0;
 
   const char *generator
     = "ldconfig " PKGVERSION RELEASE " release version " VERSION;
-  ext->sections[cache_extension_tag_generator].offset
-    = cache_extension_offset + cache_extension_size;
-  ext->sections[cache_extension_tag_generator].size = strlen (generator);
+  ext->sections[xid].tag = cache_extension_tag_generator;
+  ext->sections[xid].flags = 0;
+  ext->sections[xid].offset = generator_offset;
+  ext->sections[xid].size = strlen (generator);
+
+  if (hwcaps_count > 0)
+    {
+      ++xid;
+      ext->sections[xid].tag = cache_extension_tag_glibc_hwcaps;
+      ext->sections[xid].flags = 0;
+      ext->sections[xid].offset = hwcaps_offset;
+      ext->sections[xid].size = hwcaps_size;
+    }
+
+  ++xid;
+  ext->count = xid;
+  assert (xid <= cache_extension_count);
 
-  if (write (fd, ext, cache_extension_size) != cache_extension_size
+  size_t ext_size = (offsetof (struct cache_extension, sections)
+		     + xid * sizeof (struct cache_extension_section));
+  if (write (fd, ext, ext_size) != ext_size
+      || write (fd, hwcaps_array, hwcaps_size) != hwcaps_size
       || write (fd, generator, strlen (generator)) != strlen (generator))
     error (EXIT_FAILURE, errno, _("Writing of cache extension data failed"));
 
@@ -373,6 +553,8 @@ save_cache (const char *cache_name)
 {
   /* The cache entries are sorted already, save them in this order. */
 
+  assign_glibc_hwcaps_indices ();
+
   struct cache_entry *entry;
   /* Number of cache entries.  */
   int cache_entry_count = 0;
@@ -474,7 +656,11 @@ save_cache (const char *cache_name)
 	     struct.  */
 	  file_entries_new->libs[idx_new].flags = entry->flags;
 	  file_entries_new->libs[idx_new].osversion = entry->osversion;
-	  file_entries_new->libs[idx_new].hwcap = entry->hwcap;
+	  if (entry->hwcaps == NULL)
+	    file_entries_new->libs[idx_new].hwcap = entry->hwcap;
+	  else
+	    file_entries_new->libs[idx_new].hwcap
+	      = DL_CACHE_HWCAP_EXTENSION | entry->hwcaps->section_index;
 	  file_entries_new->libs[idx_new].key
 	    = str_offset + entry->lib->offset;
 	  file_entries_new->libs[idx_new].value
@@ -554,7 +740,7 @@ save_cache (const char *cache_name)
       /* Align file position to 4.  */
       off64_t old_offset = lseek64 (fd, extension_offset, SEEK_SET);
       assert ((unsigned long long int) (extension_offset - old_offset) < 4);
-      write_extensions (fd, extension_offset);
+      write_extensions (fd, str_offset, extension_offset);
     }
 
   /* Make sure user can always read cache file */
@@ -588,8 +774,9 @@ save_cache (const char *cache_name)
 
 /* Add one library to the cache.  */
 void
-add_to_cache (const char *path, const char *lib, int flags,
-	      unsigned int osversion, uint64_t hwcap)
+add_to_cache (const char *path, const char *filename, const char *soname,
+	      int flags, unsigned int osversion, uint64_t hwcap,
+	      struct glibc_hwcaps_subdirectory *hwcaps)
 {
   struct cache_entry *new_entry = xmalloc (sizeof (*new_entry));
 
@@ -597,11 +784,11 @@ add_to_cache (const char *path, const char *lib, int flags,
   {
     /* Use a small, on-stack buffer in most cases.  */
     char buf[200];
-    int ret = snprintf (buf, sizeof (buf), "%s/%s", path, lib);
+    int ret = snprintf (buf, sizeof (buf), "%s/%s", path, filename);
     if (ret < 0 || ret >= sizeof (buf) - 1)
       {
 	char *p;
-	if (asprintf (&p, "%s/%s", path, lib) < 0)
+	if (asprintf (&p, "%s/%s", path, filename) < 0)
 	  error (EXIT_FAILURE, errno, _("Could not create library path"));
 	path_interned = stringtable_intern (&strings, p);
 	free (p);
@@ -610,13 +797,20 @@ add_to_cache (const char *path, const char *lib, int flags,
       path_interned = stringtable_intern (&strings, buf);
   }
 
-  new_entry->lib = stringtable_intern (&strings, lib);
+  new_entry->lib = stringtable_intern (&strings, soname);
   new_entry->path = path_interned;
   new_entry->flags = flags;
   new_entry->osversion = osversion;
   new_entry->hwcap = hwcap;
+  new_entry->hwcaps = hwcaps;
   new_entry->bits_hwcap = 0;
 
+  if (hwcaps != NULL)
+    {
+      assert (hwcap == 0);
+      hwcaps->used = true;
+    }
+
   /* Count the number of bits set in the masked value.  */
   for (size_t i = 0;
        (~((1ULL << i) - 1) & hwcap) != 0 && i < 8 * sizeof (hwcap); ++i)
diff --git a/elf/ldconfig.c b/elf/ldconfig.c
index 3768267bac..3136601de7 100644
--- a/elf/ldconfig.c
+++ b/elf/ldconfig.c
@@ -16,6 +16,7 @@
    along with this program; if not, see <https://www.gnu.org/licenses/>.  */
 
 #define PROCINFO_CLASS static
+#include <assert.h>
 #include <alloca.h>
 #include <argp.h>
 #include <dirent.h>
@@ -41,6 +42,7 @@
 
 #include <ldconfig.h>
 #include <dl-cache.h>
+#include <dl-hwcaps.h>
 
 #include <dl-procinfo.h>
 
@@ -85,6 +87,10 @@ struct dir_entry
   dev_t dev;
   const char *from_file;
   int from_line;
+
+  /* Non-NULL for subdirectories under a glibc-hwcaps subdirectory.  */
+  struct glibc_hwcaps_subdirectory *hwcaps;
+
   struct dir_entry *next;
 };
 
@@ -339,17 +345,20 @@ new_sub_entry (const struct dir_entry *entry, const char *path,
   new_entry->from_line = entry->from_line;
   new_entry->path = xstrdup (path);
   new_entry->flag = entry->flag;
+  new_entry->hwcaps = NULL;
   new_entry->next = NULL;
   new_entry->ino = st->st_ino;
   new_entry->dev = st->st_dev;
   return new_entry;
 }
 
-/* Add a single directory entry.  */
-static void
+/* Add a single directory entry.  Return true if the directory is
+   actually added (because it is not a duplicate).  */
+static bool
 add_single_dir (struct dir_entry *entry, int verbose)
 {
   struct dir_entry *ptr, *prev;
+  bool added = true;
 
   ptr = dir_entries;
   prev = ptr;
@@ -369,6 +378,7 @@ add_single_dir (struct dir_entry *entry, int verbose)
 	  ptr->flag = entry->flag;
 	  free (entry->path);
 	  free (entry);
+	  added = false;
 	  break;
 	}
       prev = ptr;
@@ -379,6 +389,73 @@ add_single_dir (struct dir_entry *entry, int verbose)
     dir_entries = entry;
   else if (ptr == NULL)
     prev->next = entry;
+  return added;
+}
+
+/* Check if PATH contains a "glibc-hwcaps" subdirectory.  If so, queue
+   its subdirectories for glibc-hwcaps processing.  */
+static void
+add_glibc_hwcaps_subdirectories (struct dir_entry *entry, const char *path)
+{
+  /* glibc-hwcaps subdirectories do not nest.  */
+  assert (entry->hwcaps == NULL);
+
+  char *glibc_hwcaps;
+  if (asprintf (&glibc_hwcaps, "%s/" GLIBC_HWCAPS_SUBDIRECTORY, path) < 0)
+    error (EXIT_FAILURE, errno, _("Could not form glibc-hwcaps path"));
+
+  DIR *dir = opendir (glibc_hwcaps);
+  if (dir != NULL)
+    {
+      while (true)
+	{
+	  errno = 0;
+	  struct dirent64 *e = readdir64 (dir);
+	  if (e == NULL)
+	    {
+	      if (errno == 0)
+		break;
+	      else
+		error (EXIT_FAILURE, errno, _("Listing directory %s"), path);
+	    }
+
+	  /* Ignore hidden subdirectories, including "." and "..", and
+	     regular files.  File names containing a ':' cannot be
+	     looked up by the dynamic loader, so skip those as
+	     well.  */
+	  if (e->d_name[0] == '.' || e->d_type == DT_REG
+	      || strchr (e->d_name, ':') != NULL)
+	    continue;
+
+	  /* See if this entry eventually resolves to a directory.  */
+	  struct stat64 st;
+	  if (fstatat64 (dirfd (dir), e->d_name, &st, 0) < 0)
+	    /* Ignore unreadable entries.  */
+	    continue;
+
+	  if (S_ISDIR (st.st_mode))
+	    {
+	      /* This is a directory, so it needs to be scanned for
+		 libraries, associated with the hwcaps implied by the
+		 subdirectory name.  */
+	      char *new_path;
+	      if (asprintf (&new_path, "%s/" GLIBC_HWCAPS_SUBDIRECTORY "/%s",
+			    /* Use non-canonicalized path here.  */
+			    entry->path, e->d_name) < 0)
+		error (EXIT_FAILURE, errno,
+		       _("Could not form glibc-hwcaps path"));
+	      struct dir_entry *new_entry = new_sub_entry (entry, new_path,
+							   &st);
+	      free (new_path);
+	      new_entry->hwcaps = new_glibc_hwcaps_subdirectory (e->d_name);
+	      add_single_dir (new_entry, 0);
+	    }
+	}
+
+      closedir (dir);
+    }
+
+  free (glibc_hwcaps);
 }
 
 /* Add one directory to the list of directories to process.  */
@@ -387,6 +464,7 @@ add_dir_1 (const char *line, const char *from_file, int from_line)
 {
   unsigned int i;
   struct dir_entry *entry = xmalloc (sizeof (struct dir_entry));
+  entry->hwcaps = NULL;
   entry->next = NULL;
 
   entry->from_file = strdup (from_file);
@@ -444,7 +522,9 @@ add_dir_1 (const char *line, const char *from_file, int from_line)
       entry->ino = stat_buf.st_ino;
       entry->dev = stat_buf.st_dev;
 
-      add_single_dir (entry, 1);
+      if (add_single_dir (entry, 1))
+	/* Add glibc-hwcaps subdirectories if present.  */
+	add_glibc_hwcaps_subdirectories (entry, path);
     }
 
   if (opt_chroot)
@@ -696,15 +776,27 @@ struct dlib_entry
 static void
 search_dir (const struct dir_entry *entry)
 {
-  uint64_t hwcap = path_hwcap (entry->path);
-  if (opt_verbose)
+  uint64_t hwcap;
+  if (entry->hwcaps == NULL)
     {
-      if (hwcap != 0)
-	printf ("%s: (hwcap: %#.16" PRIx64 ")", entry->path, hwcap);
-      else
-	printf ("%s:", entry->path);
-      printf (_(" (from %s:%d)\n"), entry->from_file, entry->from_line);
+      hwcap = path_hwcap (entry->path);
+      if (opt_verbose)
+	{
+	  if (hwcap != 0)
+	    printf ("%s: (hwcap: %#.16" PRIx64 ")", entry->path, hwcap);
+	  else
+	    printf ("%s:", entry->path);
+	}
     }
+  else
+    {
+      hwcap = 0;
+      if (opt_verbose)
+	printf ("%s: (hwcap: \"%s\")", entry->path,
+		glibc_hwcaps_subdirectory_name (entry->hwcaps));
+    }
+  if (opt_verbose)
+    printf (_(" (from %s:%d)\n"), entry->from_file, entry->from_line);
 
   char *dir_name;
   char *real_file_name;
@@ -746,13 +838,15 @@ search_dir (const struct dir_entry *entry)
 	  && direntry->d_type != DT_DIR)
 	continue;
       /* Does this file look like a shared library or is it a hwcap
-	 subdirectory?  The dynamic linker is also considered as
+	 subdirectory (if not already processing a glibc-hwcaps
+	 subdirectory)?  The dynamic linker is also considered as
 	 shared library.  */
       if (((strncmp (direntry->d_name, "lib", 3) != 0
 	    && strncmp (direntry->d_name, "ld-", 3) != 0)
 	   || strstr (direntry->d_name, ".so") == NULL)
 	  && (direntry->d_type == DT_REG
-	      || !is_hwcap_platform (direntry->d_name)))
+	      || (entry->hwcaps == NULL
+		  && !is_hwcap_platform (direntry->d_name))))
 	continue;
 
       size_t len = strlen (direntry->d_name);
@@ -838,7 +932,10 @@ search_dir (const struct dir_entry *entry)
       else
 	is_dir = S_ISDIR (lstat_buf.st_mode);
 
-      if (is_dir && is_hwcap_platform (direntry->d_name))
+      /* No descending into subdirectories if this directory is a
+	 glibc-hwcaps subdirectory (which are not recursive).  */
+      if (entry->hwcaps == NULL
+	  && is_dir && is_hwcap_platform (direntry->d_name))
 	{
 	  if (!is_link
 	      && direntry->d_type != DT_UNKNOWN
@@ -1029,13 +1126,31 @@ search_dir (const struct dir_entry *entry)
   struct dlib_entry *dlib_ptr;
   for (dlib_ptr = dlibs; dlib_ptr != NULL; dlib_ptr = dlib_ptr->next)
     {
-      /* Don't create links to links.  */
-      if (dlib_ptr->is_link == 0)
-	create_links (dir_name, entry->path, dlib_ptr->name,
-		      dlib_ptr->soname);
+      /* The cached file name is the soname for non-glibc-hwcaps
+	 subdirectories (relying on symbolic links; this helps with
+	 library updates that change the file name), and the actual
+	 file for glibc-hwcaps subdirectories.  */
+      const char *filename;
+      if (entry->hwcaps == NULL)
+	{
+	  /* Don't create links to links.  */
+	  if (dlib_ptr->is_link == 0)
+	    create_links (dir_name, entry->path, dlib_ptr->name,
+			  dlib_ptr->soname);
+	  filename = dlib_ptr->soname;
+	}
+      else
+	{
+	  /* Do not create links in glibc-hwcaps subdirectories, but
+	     still log the cache addition.  */
+	  if (opt_verbose)
+	    printf ("\t%s -> %s\n", dlib_ptr->soname, dlib_ptr->name);
+	  filename = dlib_ptr->name;
+	}
       if (opt_build_cache)
-	add_to_cache (entry->path, dlib_ptr->soname, dlib_ptr->flag,
-		      dlib_ptr->osversion, hwcap);
+	add_to_cache (entry->path, filename, dlib_ptr->soname,
+		      dlib_ptr->flag, dlib_ptr->osversion,
+		      hwcap, entry->hwcaps);
     }
 
   /* Free all resources.  */
diff --git a/sysdeps/generic/dl-cache.h b/sysdeps/generic/dl-cache.h
index fec209509d..66b0312ac1 100644
--- a/sysdeps/generic/dl-cache.h
+++ b/sysdeps/generic/dl-cache.h
@@ -81,7 +81,6 @@ struct cache_file
 #define CACHE_VERSION "1.1"
 #define CACHEMAGIC_VERSION_NEW CACHEMAGIC_NEW CACHE_VERSION
 
-
 struct file_entry_new
 {
   union
@@ -99,6 +98,23 @@ struct file_entry_new
   uint64_t hwcap;		/* Hwcap entry.	 */
 };
 
+/* This bit in the hwcap field of struct file_entry_new indicates that
+   the lower 32 bits contain an index into the
+   cache_extension_tag_glibc_hwcaps section.  Older glibc versions do
+   not know about this HWCAP bit, so they will ignore these
+   entries.  */
+#define DL_CACHE_HWCAP_EXTENSION (1ULL << 62)
+
+/* Return true if the ENTRY->hwcap value indicates that
+   DL_CACHE_HWCAP_EXTENSION is used.  */
+static inline bool
+dl_cache_hwcap_extension (struct file_entry_new *entry)
+{
+  /* If DL_CACHE_HWCAP_EXTENSION is set, but other bits as well, this
+     is a different kind of extension.  */
+  return (entry->hwcap >> 32) == (DL_CACHE_HWCAP_EXTENSION >> 32);
+}
+
 /* See flags member of struct cache_file_new below.  */
 enum
   {
@@ -161,6 +177,17 @@ enum cache_extension_tag
       cache file.  */
    cache_extension_tag_generator,
 
+   /* glibc-hwcaps subdirectory information.  An array of uint32_t
+      values, which are indices into the string table.  The strings
+      are sorted lexicographically (according to strcmp).  The extra
+      level of indirection (instead of using string table indices
+      directly) allows the dynamic loader to compute the preference
+      order of the hwcaps names more efficiently.
+
+      For this section, 4-byte alignment is required, and the section
+      size must be a multiple of 4.  */
+   cache_extension_tag_glibc_hwcaps,
+
    /* Total number of known cache extension tags.  */
    cache_extension_count
   };
@@ -215,6 +242,27 @@ struct cache_extension_all_loaded
   struct cache_extension_loaded sections[cache_extension_count];
 };
 
+/* Performs basic data validation based on section tag, and removes
+   the sections which are invalid.  */
+static void
+cache_extension_verify (struct cache_extension_all_loaded *loaded)
+{
+  {
+    /* Section must not be empty, it must be aligned at 4 bytes, and
+       the size must be a multiple of 4.  */
+    struct cache_extension_loaded *hwcaps
+      = &loaded->sections[cache_extension_tag_glibc_hwcaps];
+    if (hwcaps->size == 0
+	|| ((uintptr_t) hwcaps->base % 4) != 0
+	|| (hwcaps->size % 4) != 0)
+      {
+	hwcaps->base = NULL;
+	hwcaps->size = 0;
+	hwcaps->flags = 0;
+      }
+  }
+}
+
 static bool __attribute__ ((unused))
 cache_extension_load (const struct cache_file_new *cache,
 		      const void *file_base, size_t file_size,
@@ -261,6 +309,7 @@ cache_extension_load (const struct cache_file_new *cache,
       loaded->sections[tag].size = ext->sections[i].size;
       loaded->sections[tag].flags = ext->sections[i].flags;
     }
+  cache_extension_verify (loaded);
   return true;
 }
 
diff --git a/sysdeps/generic/ldconfig.h b/sysdeps/generic/ldconfig.h
index b64aab0064..30a76481aa 100644
--- a/sysdeps/generic/ldconfig.h
+++ b/sysdeps/generic/ldconfig.h
@@ -57,8 +57,22 @@ extern void init_cache (void);
 
 extern void save_cache (const char *cache_name);
 
-extern void add_to_cache (const char *path, const char *lib, int flags,
-			  unsigned int osversion, uint64_t hwcap);
+struct glibc_hwcaps_subdirectory;
+
+/* Return a struct describing the subdirectory for NAME.  Reuse an
+   existing struct if it exists.  */
+struct glibc_hwcaps_subdirectory *new_glibc_hwcaps_subdirectory
+  (const char *name);
+
+/* Returns the name that was specified when
+   add_glibc_hwcaps_subdirectory was called.  */
+const char *glibc_hwcaps_subdirectory_name
+  (struct glibc_hwcaps_subdirectory *);
+
+extern void add_to_cache (const char *path, const char *filename,
+			  const char *soname,
+			  int flags, unsigned int osversion, uint64_t hwcap,
+			  struct glibc_hwcaps_subdirectory *);
 
 extern void init_aux_cache (void);
 
-- 
2.25.4

[PATCH 30/30] elf: Add glibc-hwcaps subdirectory support to ld.so cache processing

[Florian Weimer]

This recognizes the DL_CACHE_HWCAP_EXTENSION flag and picks up
the supported cache entry with the highest priority.
---
 elf/dl-cache.c  | 182 +++++++++++++++++++++++++++++++++++++++++++++++-
 elf/dl-hwcaps.c |  78 +++++++++++++++++++++
 elf/dl-hwcaps.h |  19 +++++
 3 files changed, 276 insertions(+), 3 deletions(-)

diff --git a/elf/dl-cache.c b/elf/dl-cache.c
index 02c46ffb0c..4714119974 100644
--- a/elf/dl-cache.c
+++ b/elf/dl-cache.c
@@ -35,6 +35,132 @@ static struct cache_file *cache;
 static struct cache_file_new *cache_new;
 static size_t cachesize;
 
+#ifdef SHARED
+/* This is used to cache the priorities of glibc-hwcaps
+   subdirectories.  The elements of _dl_cache_priorities correspond to
+   the strings in the cache_extension_tag_glibc_hwcaps section.  */
+static uint32_t *glibc_hwcaps_priorities;
+static uint32_t glibc_hwcaps_priorities_length;
+static uint32_t glibc_hwcaps_priorities_allocated;
+
+/* True if the full malloc was used to allocated the array.  */
+static bool glibc_hwcaps_priorities_malloced;
+
+/* Deallocate the glibc_hwcaps_priorities array.  */
+static void
+glibc_hwcaps_priorities_free (void)
+{
+  /* When the minimal malloc is in use, free does not do anything,
+     so it does not make sense to call it.  */
+  if (glibc_hwcaps_priorities_malloced)
+    free (glibc_hwcaps_priorities);
+  glibc_hwcaps_priorities = NULL;
+  glibc_hwcaps_priorities_allocated = 0;
+}
+
+/* Return the priority of the cache_extension_tag_glibc_hwcaps section
+   entry at INDEX.  Zero means do not use.  Otherwise, lower values
+   indicate greater preference.  */
+static uint32_t __attribute__ ((noinline, noclone))
+glibc_hwcaps_priority (uint32_t index)
+{
+  /* Using a zero-length array as an indicator that nothing has been
+     loaded is not a problem: It does not lead to repeated
+     initialization attempts because caches without an extension
+     section are processed without calling this function (unless the
+     file is corrupted).  */
+  if (glibc_hwcaps_priorities_length == 0)
+    {
+      struct cache_extension_all_loaded ext;
+      if (!cache_extension_load (cache_new, cache, cachesize, &ext))
+	return 0;
+
+      uint32_t length
+	= (ext.sections[cache_extension_tag_glibc_hwcaps].size
+	   / sizeof (uint32_t));
+      if (length > glibc_hwcaps_priorities_allocated)
+	{
+	  glibc_hwcaps_priorities_free ();
+
+	  glibc_hwcaps_priorities = malloc (length * sizeof (uint32_t));
+	  if (glibc_hwcaps_priorities == NULL)
+	    /* Disable hwcaps on memory allocation error.  */
+	    return 0;
+
+	  glibc_hwcaps_priorities_allocated = length;
+	  glibc_hwcaps_priorities_malloced = __rtld_malloc_is_full ();
+	}
+
+      /* Compute the priorities for the subdirectories by merging the
+	 array in the cache with the dl_hwcaps_priorities array.  */
+      const uint32_t *left
+	= ext.sections[cache_extension_tag_glibc_hwcaps].base;
+      const uint32_t *left_end = left + length;
+      struct dl_hwcaps_priority *right = _dl_hwcaps_priorities;
+      struct dl_hwcaps_priority *right_end
+	= right + _dl_hwcaps_priorities_length;
+      uint32_t *result = glibc_hwcaps_priorities;
+
+      while (left < left_end && right < right_end)
+	{
+	  uint32_t string_table_index = *left;
+	  if (string_table_index < cachesize)
+	    {
+	      const char *left_name
+		= (const char *) cache + string_table_index;
+	      uint32_t left_name_length = strlen (left_name);
+	      uint32_t to_compare;
+	      if (left_name_length < right->name_length)
+		to_compare = left_name_length;
+	      else
+		to_compare = right->name_length;
+	      int cmp = memcmp (left_name, right->name, to_compare);
+	      if (cmp == 0)
+		{
+		  if (left_name_length < right->name_length)
+		    cmp = -1;
+		  else if (left_name_length > right->name_length)
+		    cmp = 1;
+		}
+	      if (cmp == 0)
+		{
+		  *result = right->priority;
+		  ++result;
+		  ++left;
+		  ++right;
+		}
+	      else if (cmp < 0)
+		{
+		  *result = 0;
+		  ++result;
+		  ++left;
+		}
+	      else
+		++right;
+	    }
+	  else
+	    {
+	      *result = 0;
+	      ++result;
+	    }
+	}
+      while (left < left_end)
+	{
+	  *result = 0;
+	  ++result;
+	  ++left;
+	}
+
+      glibc_hwcaps_priorities_length = length;
+    }
+
+  if (index < glibc_hwcaps_priorities_length)
+    return glibc_hwcaps_priorities[index];
+  else
+    return 0;
+}
+#endif /* SHARED */
+
 /* True if PTR is a valid string table index.  */
 static inline bool
 _dl_cache_verify_ptr (uint32_t ptr, size_t string_table_size)
@@ -74,6 +200,9 @@ search_cache (const char *string_table, uint32_t string_table_size,
   int left = 0;
   int right = nlibs - 1;
   const char *best = NULL;
+#ifdef SHARED
+  uint32_t best_priority = 0;
+#endif
 
   while (left <= right)
     {
@@ -129,6 +258,11 @@ search_cache (const char *string_table, uint32_t string_table_size,
 		{
 		  if (best == NULL || flags == GLRO (dl_correct_cache_id))
 		    {
+		      /* Named/extension hwcaps get slightly different
+			 treatment: We keep searching for a better
+			 match.  */
+		      bool named_hwcap = false;
+
 		      if (entry_size >= sizeof (struct file_entry_new))
 			{
 			  /* The entry is large enough to include
@@ -136,7 +270,18 @@ search_cache (const char *string_table, uint32_t string_table_size,
 			  struct file_entry_new *libnew
 			    = (struct file_entry_new *) lib;
 
-			  if (libnew->hwcap & hwcap_exclude)
+#ifdef SHARED
+			  named_hwcap = dl_cache_hwcap_extension (libnew);
+#endif
+
+			  /* The entries with named/extension hwcaps
+			     have been exhausted.  Return the best
+			     match encountered so far if there is
+			     one.  */
+			  if (!named_hwcap && best != NULL)
+			    break;
+
+			  if ((libnew->hwcap & hwcap_exclude) && !named_hwcap)
 			    continue;
 			  if (GLRO (dl_osversion)
 			      && libnew->osversion > GLRO (dl_osversion))
@@ -146,14 +291,41 @@ search_cache (const char *string_table, uint32_t string_table_size,
 			      && ((libnew->hwcap & _DL_HWCAP_PLATFORM)
 				  != platform))
 			    continue;
+
+#ifdef SHARED
+			  /* For named hwcaps, determine the priority
+			     and see if beats what has been found so
+			     far.  */
+			  if (named_hwcap)
+			    {
+			      uint32_t entry_priority
+				= glibc_hwcaps_priority (libnew->hwcap);
+			      if (entry_priority == 0)
+				/* Not usable at all.  Skip.  */
+				continue;
+			      else if (best == NULL
+				       || entry_priority < best_priority)
+				/* This entry is of higher priority
+				   than the previous one, or it is the
+				   first entry.  */
+				best_priority = entry_priority;
+			      else
+				/* An entry has already been found,
+				   but it is a better match.  */
+				continue;
+			    }
+#endif /* SHARED */
 			}
 
 		      best = string_table + lib->value;
 
-		      if (flags == GLRO (dl_correct_cache_id))
+		      if (flags == GLRO (dl_correct_cache_id)
+			  && !named_hwcap)
 			/* We've found an exact match for the shared
 			   object and no general `ELF' release.  Stop
-			   searching.  */
+			   searching, but not if a named (extension)
+			   hwcap is used.  In this case, an entry with
+			   a higher priority may come up later.  */
 			break;
 		    }
 		}
@@ -346,5 +518,9 @@ _dl_unload_cache (void)
       __munmap (cache, cachesize);
       cache = NULL;
     }
+#ifdef SHARED
+  /* This marks the glibc_hwcaps_priorities array as out-of-date.  */
+  glibc_hwcaps_priorities_length = 0;
+#endif
 }
 #endif
diff --git a/elf/dl-hwcaps.c b/elf/dl-hwcaps.c
index 4de94759a2..cb53337025 100644
--- a/elf/dl-hwcaps.c
+++ b/elf/dl-hwcaps.c
@@ -89,6 +89,81 @@ copy_hwcaps (struct copy_hwcaps *target, const char *hwcaps,
     }
 }
 
+struct dl_hwcaps_priority *_dl_hwcaps_priorities;
+uint32_t _dl_hwcaps_priorities_length;
+
+/* Allocate _dl_hwcaps_priorities and fill it with data.  */
+static void
+compute_priorities (size_t total_count, const char *prepend,
+		    int32_t bitmask, const char *mask)
+{
+  _dl_hwcaps_priorities = malloc (total_count
+				  * sizeof (*_dl_hwcaps_priorities));
+  if (_dl_hwcaps_priorities == NULL)
+    _dl_signal_error (ENOMEM, NULL, NULL,
+		      N_("cannot create HWCAP priorities"));
+  _dl_hwcaps_priorities_length = total_count;
+
+  /* First the prepended subdirectories.  */
+  size_t i = 0;
+  {
+    struct dl_hwcaps_split sp;
+    _dl_hwcaps_split_init (&sp, prepend);
+    while (_dl_hwcaps_split (&sp))
+      {
+	_dl_hwcaps_priorities[i].name = sp.segment;
+	_dl_hwcaps_priorities[i].name_length = sp.length;
+	_dl_hwcaps_priorities[i].priority = i + 1;
+	++i;
+      }
+  }
+
+  /* Then the built-in subdirectories that are actually active.  */
+  {
+    struct dl_hwcaps_split_masked sp;
+    _dl_hwcaps_split_masked_init (&sp, _dl_hwcaps_subdirs, bitmask, mask);
+    while (_dl_hwcaps_split_masked (&sp))
+      {
+	_dl_hwcaps_priorities[i].name = sp.split.segment;
+	_dl_hwcaps_priorities[i].name_length = sp.split.length;
+	_dl_hwcaps_priorities[i].priority = i + 1;
+	++i;
+      }
+  }
+  assert (i == total_count);
+}
+
+/* Sort the _dl_hwcaps_priorities array by name.  */
+static void
+sort_priorities_by_name (void)
+{
+  /* Insertion sort.  There is no need to link qsort into the dynamic
+     loader for such a short array.  */
+  for (size_t i = 1; i < _dl_hwcaps_priorities_length; ++i)
+    for (size_t j = i; j > 0; --j)
+      {
+	struct dl_hwcaps_priority *previous = _dl_hwcaps_priorities + j - 1;
+	struct dl_hwcaps_priority *current = _dl_hwcaps_priorities + j;
+
+	/* Bail out if current is greater or equal to the previous
+	   value.  */
+	uint32_t to_compare;
+	if (current->name_length < previous->name_length)
+	  to_compare = current->name_length;
+	else
+	  to_compare = previous->name_length;
+	int cmp = memcmp (current->name, previous->name, to_compare);
+	if (cmp >= 0
+	    || (cmp == 0 && current->name_length >= previous->name_length))
+	  break;
+
+	/* Swap *previous and *current.  */
+	struct dl_hwcaps_priority tmp = *previous;
+	*previous = *current;
+	*current = tmp;
+      }
+}
+
 /* Return an array of useful/necessary hardware capability names.  */
 const struct r_strlenpair *
 _dl_important_hwcaps (const char *glibc_hwcaps_prepend,
@@ -111,6 +186,9 @@ _dl_important_hwcaps (const char *glibc_hwcaps_prepend,
   count_hwcaps (&hwcaps_counts, glibc_hwcaps_prepend, -1, NULL);
   count_hwcaps (&hwcaps_counts, _dl_hwcaps_subdirs, hwcaps_subdirs_active,
 		glibc_hwcaps_mask);
+  compute_priorities (hwcaps_counts.count, glibc_hwcaps_prepend,
+		      hwcaps_subdirs_active, glibc_hwcaps_mask);
+  sort_priorities_by_name ();
 
   /* Each hwcaps subdirectory has a GLIBC_HWCAPS_PREFIX string prefix
      and a "/" suffix once stored in the result.  */
diff --git a/elf/dl-hwcaps.h b/elf/dl-hwcaps.h
index a6453f15f3..4b4c143854 100644
--- a/elf/dl-hwcaps.h
+++ b/elf/dl-hwcaps.h
@@ -110,4 +110,23 @@ extern const char _dl_hwcaps_subdirs[] attribute_hidden;
    bitmask.  */
 int32_t _dl_hwcaps_subdirs_active (void) attribute_hidden;
 
+/* Pre-computed glibc-hwcaps subdirectory priorities.  Used in
+   dl-cache.c to quickly find the proprities for the stored HWCAP
+   names.  */
+struct dl_hwcaps_priority
+{
+  /* The name consists of name_length bytes at name (not necessarily
+     null-terminated).  */
+  const char *name;
+  uint32_t name_length;
+
+  /* Priority of this name.  A positive number.  */
+  uint32_t priority;
+};
+
+/* Pre-computed hwcaps priorities.  Set up by
+   _dl_important_hwcaps.  */
+extern struct dl_hwcaps_priority *_dl_hwcaps_priorities attribute_hidden;
+extern uint32_t _dl_hwcaps_priorities_length attribute_hidden;
+
 #endif /* _DL_HWCAPS_H */
-- 
2.25.4

Re: [PATCH 01/30] elf: Include <stdbool.h> in <dl-tunables.h> because bool is used

[Tulio Magno Quites Machado Filho]

Florian Weimer via Libc-alpha <libc-alpha@sourceware.org> writes:

> diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
> index 969e50327b..f05eb50c2f 100644
> --- a/elf/dl-tunables.h
> +++ b/elf/dl-tunables.h
> @@ -21,6 +21,8 @@
>  #ifndef _TUNABLES_H_
>  #define _TUNABLES_H_
>  
> +#include <stdbool.h>
> +

LGTM.

I think patches #1 and #2 should go in regardless of the other patches.

-- 
Tulio Magno

Re: [PATCH 01/30] elf: Include <stdbool.h> in <dl-tunables.h> because bool is used

[Florian Weimer]

* Tulio Magno Quites Machado Filho:

> Florian Weimer via Libc-alpha <libc-alpha@sourceware.org> writes:
>
>> diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
>> index 969e50327b..f05eb50c2f 100644
>> --- a/elf/dl-tunables.h
>> +++ b/elf/dl-tunables.h
>> @@ -21,6 +21,8 @@
>>  #ifndef _TUNABLES_H_
>>  #define _TUNABLES_H_
>>  
>> +#include <stdbool.h>
>> +
>
> LGTM.

Thanks.

> I think patches #1 and #2 should go in regardless of the other patches.

Yes, I think the ld.so --help stuff is independently useful.

Florian

Re: [PATCH 19/30] x86_64: Add glibc-hwcaps support

[Florian Weimer]

* Florian Weimer via Libc-alpha:

> Details of the supported CPU flags and the names are still subject to
> changes.
> +int32_t
> +_dl_hwcaps_subdirs_active (void)
> +{
> +  const struct cpu_features *cpu_features = __get_cpu_features ();
> +  int32_t result = 0;
> +  int32_t bit = 1 << 3;
> +
> +  /* Test in reverse preference order.  */
> +
> +  /* x86-100.  */
> +  if (!(CPU_FEATURES_ARCH_P (cpu_features, CMPXCHG16B_Usable)
> +        && CPU_FEATURES_ARCH_P (cpu_features, POPCNT_Usable)
> +        && CPU_FEATURES_ARCH_P (cpu_features, SSE3_Usable)
> +        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_1_Usable)
> +        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_2_Usable)
> +        && CPU_FEATURES_ARCH_P (cpu_features, SSSE3_Usable)))
> +    return result;

This misuses CPU_FEATURES_ARCH_P, and the recent <cpu-features.h> update
on the master branch makes it finally fail to compile.  The version
below should fix the build and also correct the feature detection logic.

Thanks,
Florian

diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
new file mode 100644
index 0000000000..1a90854b69
--- /dev/null
+++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
@@ -0,0 +1,73 @@
+/* Architecture-specific glibc-hwcaps subdirectories.  x86 version.
+   Copyright (C) 2020 Free Software Foundation, Inc.
+   This file is part of the GNU C Library.
+
+   The GNU C Library is free software; you can redistribute it and/or
+   modify it under the terms of the GNU Lesser General Public
+   License as published by the Free Software Foundation; either
+   version 2.1 of the License, or (at your option) any later version.
+
+   The GNU C Library is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+   Lesser General Public License for more details.
+
+   You should have received a copy of the GNU Lesser General Public
+   License along with the GNU C Library; if not, see
+   <https://www.gnu.org/licenses/>.  */
+
+#include <dl-hwcaps.h>
+#include <cpu-features.h>
+
+const char _dl_hwcaps_subdirs[] = "x86-103:x86-102:x86-101:x86-100";
+
+int32_t
+_dl_hwcaps_subdirs_active (void)
+{
+  const struct cpu_features *cpu_features = __get_cpu_features ();
+  int32_t result = 0;
+  int32_t bit = 1 << 3;
+
+  /* Test in reverse preference order.  */
+
+  /* x86-100.  */
+  if (!(CPU_FEATURES_CPU_P (cpu_features, CMPXCHG16B)
+        && CPU_FEATURES_CPU_P (cpu_features, POPCNT)
+        && CPU_FEATURES_CPU_P (cpu_features, SSE3)
+        && CPU_FEATURES_CPU_P (cpu_features, SSE4_1)
+        && CPU_FEATURES_CPU_P (cpu_features, SSE4_2)
+        && CPU_FEATURES_CPU_P (cpu_features, SSSE3)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* x86-101.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX_Usable)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  /* x86-102.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX2_Usable)
+        && CPU_FEATURES_CPU_P (cpu_features, BMI1)
+        && CPU_FEATURES_CPU_P (cpu_features, BMI2)
+        && CPU_FEATURES_ARCH_P (cpu_features, F16C_Usable)
+        && CPU_FEATURES_ARCH_P (cpu_features, FMA_Usable)
+        && CPU_FEATURES_CPU_P (cpu_features, LZCNT)
+        && CPU_FEATURES_CPU_P (cpu_features, MOVBE)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+ /* x86-103.  */
+  if (!(CPU_FEATURES_ARCH_P (cpu_features, AVX512F_Usable)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512BW)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512CD)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512DQ)
+        && CPU_FEATURES_CPU_P (cpu_features, AVX512VL)))
+    return result;
+  result |= bit;
+  bit >>= 1;
+
+  return result;
+}

Re: [PATCH 02/30] elf: Include <stddef.h> (for size_t), <sys/stat.h> in <ldconfig.h>

[Carlos O'Donell]

On 6/22/20 11:13 AM, Florian Weimer via Libc-alpha wrote:
> ---
>  sysdeps/generic/ldconfig.h | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/sysdeps/generic/ldconfig.h b/sysdeps/generic/ldconfig.h
> index a8a2be7856..b64aab0064 100644
> --- a/sysdeps/generic/ldconfig.h
> +++ b/sysdeps/generic/ldconfig.h
> @@ -19,7 +19,9 @@
>  #ifndef _LDCONFIG_H
>  #define _LDCONFIG_H
>  
> +#include <stddef.h>
>  #include <stdint.h>
> +#include <sys/stat.h>
>  
>  #define FLAG_ANY			-1
>  #define FLAG_TYPE_MASK			0x00ff
> 

OK for master.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

-- 
Cheers,
Carlos.

Re: [PATCH 19/30] x86_64: Add glibc-hwcaps support

[H.J. Lu]

On Thu, Jun 25, 2020 at 3:14 AM Florian Weimer via Libc-alpha
<libc-alpha@sourceware.org> wrote:
>
> * Florian Weimer via Libc-alpha:
>
> > Details of the supported CPU flags and the names are still subject to
> > changes.
> > +int32_t
> > +_dl_hwcaps_subdirs_active (void)
> > +{
> > +  const struct cpu_features *cpu_features = __get_cpu_features ();
> > +  int32_t result = 0;
> > +  int32_t bit = 1 << 3;
> > +
> > +  /* Test in reverse preference order.  */
> > +
> > +  /* x86-100.  */
> > +  if (!(CPU_FEATURES_ARCH_P (cpu_features, CMPXCHG16B_Usable)
> > +        && CPU_FEATURES_ARCH_P (cpu_features, POPCNT_Usable)
> > +        && CPU_FEATURES_ARCH_P (cpu_features, SSE3_Usable)
> > +        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_1_Usable)
> > +        && CPU_FEATURES_ARCH_P (cpu_features, SSE4_2_Usable)
> > +        && CPU_FEATURES_ARCH_P (cpu_features, SSSE3_Usable)))
> > +    return result;
>
> This misuses CPU_FEATURES_ARCH_P, and the recent <cpu-features.h> update
> on the master branch makes it finally fail to compile.  The version
> below should fix the build and also correct the feature detection logic.
>
> Thanks,
> Florian
>
> diff --git a/sysdeps/x86_64/dl-hwcaps-subdirs.c b/sysdeps/x86_64/dl-hwcaps-subdirs.c
> new file mode 100644
> index 0000000000..1a90854b69
> --- /dev/null
> +++ b/sysdeps/x86_64/dl-hwcaps-subdirs.c
> @@ -0,0 +1,73 @@
> +/* Architecture-specific glibc-hwcaps subdirectories.  x86 version.
> +   Copyright (C) 2020 Free Software Foundation, Inc.
> +   This file is part of the GNU C Library.
> +
> +   The GNU C Library is free software; you can redistribute it and/or
> +   modify it under the terms of the GNU Lesser General Public
> +   License as published by the Free Software Foundation; either
> +   version 2.1 of the License, or (at your option) any later version.
> +
> +   The GNU C Library is distributed in the hope that it will be useful,
> +   but WITHOUT ANY WARRANTY; without even the implied warranty of
> +   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +   Lesser General Public License for more details.
> +
> +   You should have received a copy of the GNU Lesser General Public
> +   License along with the GNU C Library; if not, see
> +   <https://www.gnu.org/licenses/>.  */
> +
> +#include <dl-hwcaps.h>
> +#include <cpu-features.h>
> +
> +const char _dl_hwcaps_subdirs[] = "x86-103:x86-102:x86-101:x86-100";
>

I understand why you use digits.  But it is very cryptic.  Since it is
user visible,
how can one tell what 103 means without looking at glibc source?

-- 
H.J.

Re: [PATCH 00/30] RFC: elf: glibc-hwcaps support

[Jim Wilson]

On Mon, Jun 22, 2020 at 8:13 AM Florian Weimer via Libc-alpha
<libc-alpha@sourceware.org> wrote:
> With these changes, on a current x86-64 machine (with AVX2-level CPU
> features), you can drop a shared object into the directory
>   /usr/lib64/glibc-hwcaps/x86-102

This appears to violate the Filesystem Hierarchy Standard (FHS)
    https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html
which states that libraries can only be in /usr/lib or /usr/lib<qual>.
And it is implied but not clearly stated that <qual> is not allowed to
contain a slash.  There is an exception for applications that can have
a dir under /usr/lib or /usr/lib<qual>, but glibc isn't an
application.  Unless maybe you want to claim that ld.so is an
application and these are ld.so specific files?  But then all
libraries used by ld.so must be in there which is not true.

I'm not sure if anyone cares about strict compliance with FHS though.
Debian (and hence Ubuntu) deliberately violate it with their multiarch
scheme.  RISC-V violates it with our two level scheme for word size
and abi, e.g. /usr/lib64/lp64d for 64-bit hard-float libraries.
Though one person did care enough to report this as a bug.  I sent
email to the fhs-discuss mailing list to ask about this, but never got
a reply.  The email I sent in April is the last email on the mailing
list.  Anyways, I don't care about FHS personally, I just care that I
don't need to change the RISC-V ABI.  If this scheme is OK then maybe
the RISC-V scheme is OK too?

Jim

Re: [PATCH 00/30] RFC: elf: glibc-hwcaps support

[Florian Weimer]

* Jim Wilson:

> On Mon, Jun 22, 2020 at 8:13 AM Florian Weimer via Libc-alpha
> <libc-alpha@sourceware.org> wrote:
>> With these changes, on a current x86-64 machine (with AVX2-level CPU
>> features), you can drop a shared object into the directory
>>   /usr/lib64/glibc-hwcaps/x86-102
>
> This appears to violate the Filesystem Hierarchy Standard (FHS)
>     https://refspecs.linuxfoundation.org/FHS_3.0/fhs/index.html
> which states that libraries can only be in /usr/lib or /usr/lib<qual>.
> And it is implied but not clearly stated that <qual> is not allowed to
> contain a slash.  There is an exception for applications that can have
> a dir under /usr/lib or /usr/lib<qual>, but glibc isn't an
> application.  Unless maybe you want to claim that ld.so is an
> application and these are ld.so specific files?  But then all
> libraries used by ld.so must be in there which is not true.

I thought about this.  For use with the library path, it has to be a
subdirectory not a sibling directory, otherwise the behavior would be
really surprising.

In the end, it's no different from the "tls" directory we already search
(along with a bunch of others).  I posted a list of the subdirectories
searched on s390x:

  <https://sourceware.org/pipermail/libc-alpha/2020-May/113757.html>

At least the "glibc-hwcaps" name is architecture-independent, and we can
add it to FHS if we wanted.

Thanks,
Florian