Re: [RFC PATCH v1 1/4] Static Trampolines

[Anthony Green <green@moxielogic.com>]

Thank you for your submission, Madhavan.  This will be interesting reading!

In the meantime, it would be helpful if you submitted this as a PR on
GitHub.  This will trigger CI testing for over 20 different systems so we
can have an initial look at any build/test problems.

AG

On Tue, Nov 24, 2020 at 2:32 PM madvenka--- via Libffi-discuss <
libffi-discuss@sourceware.org> wrote:

> From: "Madhavan T. Venkataraman" <madvenka@linux.microsoft.com>
>
> Closure Trampoline Security Issue
> =================================
>
> Currently, the trampoline code used in libffi is not statically defined in
> a source file (except for MACH). The trampoline is either pre-defined
> machine code in a data buffer. Or, it is generated at runtime. In order to
> execute a trampoline, it needs to be placed in a page with executable
> permissions.
>
> Executable data pages are attack surfaces for attackers who may try to
> inject their own code into the page and contrive to have it executed. The
> security settings in a system may prevent various tricks used in user land
> to write code into a page and to have it executed somehow. On such systems,
> libffi trampolines would not be able to run.
>
> Static Trampoline
> =================
>
> To solve this problem, the trampoline code needs to be defined statically
> in a source file, compiled and placed in the text segment so it can be
> mapped and executed naturally without any tricks. However, the trampoline
> needs to be able to access the closure pointer at runtime.
>
> PC-relative data referencing
> ============================
>
> The solution implemented in this patch set uses PC-relative data
> references.
> The trampoline is mapped in a code page. Adjacent to the code page, a data
> page is mapped that contains the parameters of the trampoline:
>
>         - the closure pointer
>         - pointer to the ABI handler to jump to
>
> The trampoline code uses an offset relative to its current PC to access its
> data.
>
> Some architectures support PC-relative data references in the ISA itself.
> E.g., X64 supports RIP-relative references. For others, the PC has to
> somehow be loaded into a general purpose register to do PC-relative data
> referencing. To do this, we need to define a get_pc() kind of function and
> call it to load the PC in a desired register.
>
> There are two cases:
>
> 1. The call instruction pushes the return address on the stack.
>
>    In this case, get_pc() will extract the return address from the stack
>    and load it in the desired register and return.
>
> 2. The call instruction stores the return address in a designated register.
>
>    In this case, get_pc() will copy the return address to the desired
>    register and return.
>
> Either way, the PC next to the call instruction is obtained.
>
> Scratch register
> ================
>
> In order to do its job, the trampoline code would be required to use a
> scratch register. Depending on the ABI, there may not be a register
> available for scratch. This problem needs to be solved so that all ABIs
> will work.
>
> The trampoline will save two values on the stack:
>
>         - the closure pointer
>         - the original value of the scratch register
>
> This is what the stack will look like:
>
>         sp before trampoline ------>    --------------------
>                                         | closure pointer  |
>                                         --------------------
>                                         | scratch register |
>         sp after trampoline ------->    --------------------
>
> The ABI handler can do the following as needed by the ABI:
>
>         - the closure pointer can be loaded in a desired register
>
>         - the scratch register can be restored to its original value
>
>         - the stack pointer can be restored to its original value
>           (when the trampoline was invoked)
>
> Thus the ABI handlers will have a couple of lines of code at the very
> beginning to do this so that all ABIs will work.
>
> NOTE:
>         The documentation for this feature will contain information on:
>
>         - the name of the scratch register for each architecture
>
>         - the stack offsets at which the closure and the scratch register
>           will be copied
>
> Trampoline Table
> ================
>
> In order to reduce the trampoline memory footprint, the trampoline code
> would be defined as a code array in the text segment. This array would be
> mapped into the address space of the caller. The mapping would, therefore,
> contain a trampoline table.
>
> Adjacent to the trampoline table, there will be a data mapping that
> contains
> a parameter table, one parameter block for each trampoline. The parameter
> table will contain:
>
>         - a pointer to the closure
>         - a pointer to the ABI handler
>
> The trampoline code would finally look like this:
>
>         - Make space on the stack for the closure and the scratch register
>           by moving the stack pointer down
>         - Store the original value of the scratch register on the stack
>         - Using PC-relative reference, get the closure pointer
>         - Store the closure pointer on the stack
>         - Using PC-relative reference, get the ABI handler pointer
>         - Jump to the ABI handler
>
> Trampoline API
> ==============
>
> There is a lot of dynamic code out there. They all have the same security
> issue. Dynamic code can be re-written into static code provided the data
> required by the static code can be passed to it just like we pass the
> closure pointer to an ABI handler.
>
> So, the same trampoline functions used by libffi internally need to be
> made available to the rest of the world in the form of an API. The
> following API has been defined in this solution:
>
> int ffi_tramp_is_supported(void);
>
>         To support static trampolines, code needs to be added to each
>         architecture. Also, the feature itself can be enabled via a
>         configuration option. So, this function tells us if the feature
>         is supported and enabled in the current libffi or not.
>
> void *ffi_tramp_alloc (int flags);
>
>         Allocate a trampoline. Currently, flags are zero. An opaque
>         trampoline structure pointer is returned.
>
>         Internally, libffi manages trampoline tables and individual
>         trampolines in each table.
>
> int ffi_tramp_set_parms (void *tramp, void *target, void *data);
>
>         Initialize the parameters of a trampoline. That is, the target code
>         that the trampoline should jump to and the data that needs to be
>         passed to the target code.
>
> void *ffi_tramp_get_addr (void *tramp);
>
>         Return the address of the trampoline to invoke the trampoline with.
>         The trampoline can be invoked in one of two ways:
>
>                 - Simply branch to the trampoline address
>                 - Treat the trampoline address as a function pointer and
>                   call it.
>
>         Which method is used depends on the target code.
>
> void ffi_tramp_free (void *tramp);
>
>         Free a trampoline.
>
> Configuration
> =============
>
> A new configuration option, --enable-static-tramp has been added to enable
> the use of static trampolines.
>
> Mapping size
> ============
>
> The size of the code mapping that contains the trampoline table needs to be
> determined on a per architecture basis. If a particular architecture
> supports multiple base page sizes, then the largest base page size needs to
> be chosen. E.g., we choose 16K for ARM64.
>
> Trampoline allocation and free
> ==============================
>
> Static trampolines are allocated in ffi_closure_alloc() and freed in
> ffi_closure_free().
>
> Normally, applications use these functions. But there are some cases out
> there where the user of libffi allocates and manages its own closure
> memory. In such cases, the static trampoline API cannot be used. These
> will fall back to using legacy trampolines. The user has to make sure
> that the memory is executable.
>
> ffi_closure structure
> =====================
>
> I did not want to make any changes to the size of the closure structure for
> this feature to guarantee compatibility. But the opaque static trampoline
> handle needs to be stored in the closure. I have defined it as follows:
>
> -  char tramp[FFI_TRAMPOLINE_SIZE];
> +  union {
> +    char tramp[FFI_TRAMPOLINE_SIZE];
> +    void *ftramp;
> +  };
>
> If static trampolines are used, then tramp[] is not needed to store a
> dynamic trampoline. That space can be reused to store the handle. Hence,
> the union.
>
> Signed-off-by: Madhavan T. Venkataraman <madvenka@linux.microsoft.com>
> ---
>  Makefile.am                                 |   3 +-
>  configure.ac                                |   7 +
>  include/ffi.h.in                            |  13 +-
>  include/ffi_common.h                        |   4 +
>  libffi.map.in                               |  11 +
>  src/closures.c                              |  54 +-
>  src/tramp.c                                 | 563 ++++++++++++++++++++
>  testsuite/libffi.closures/closure_loc_fn0.c |   3 +
>  8 files changed, 654 insertions(+), 4 deletions(-)
>  create mode 100644 src/tramp.c
>
> diff --git a/Makefile.am b/Makefile.am
> index 7654bf5..1b18198 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -38,7 +38,8 @@ toolexeclib_LTLIBRARIES = libffi.la
>  noinst_LTLIBRARIES = libffi_convenience.la
>
>  libffi_la_SOURCES = src/prep_cif.c src/types.c \
> -               src/raw_api.c src/java_raw_api.c src/closures.c
> +               src/raw_api.c src/java_raw_api.c src/closures.c \
> +               src/tramp.c
>
>  if FFI_DEBUG
>  libffi_la_SOURCES += src/debug.c
> diff --git a/configure.ac b/configure.ac
> index 790274e..898ede6 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -360,6 +360,13 @@ AC_ARG_ENABLE(raw-api,
>      AC_DEFINE(FFI_NO_RAW_API, 1, [Define this if you do not want support
> for the raw API.])
>    fi)
>
> +AC_ARG_ENABLE(static-tramp,
> +[  --enable-static-tramp        use statically defined trampolines],
> +  if test "$enable_static_tramp" = "yes"; then
> +    AC_DEFINE(FFI_EXEC_STATIC_TRAMP, 1,
> +      [Define this if you want to use statically defined trampolines.])
> +  fi)
> +
>  AC_ARG_ENABLE(purify-safety,
>  [  --enable-purify-safety  purify-safe mode],
>    if test "$enable_purify_safety" = "yes"; then
> diff --git a/include/ffi.h.in b/include/ffi.h.in
> index 38885b0..c6a21d9 100644
> --- a/include/ffi.h.in
> +++ b/include/ffi.h.in
> @@ -310,7 +310,10 @@ typedef struct {
>    void *trampoline_table;
>    void *trampoline_table_entry;
>  #else
> -  char tramp[FFI_TRAMPOLINE_SIZE];
> +  union {
> +    char tramp[FFI_TRAMPOLINE_SIZE];
> +    void *ftramp;
> +  };
>  #endif
>    ffi_cif   *cif;
>    void     (*fun)(ffi_cif*,void*,void**,void*);
> @@ -457,6 +460,14 @@ FFI_API void ffi_call_go (ffi_cif *cif, void
> (*fn)(void), void *rvalue,
>
>  #endif /* FFI_GO_CLOSURES */
>
> +/* ---- Static Trampoline Definitions
> -------------------------------------- */
> +
> +FFI_API int ffi_tramp_is_supported(void);
> +FFI_API void *ffi_tramp_alloc (int flags);
> +FFI_API int ffi_tramp_set_parms (void *tramp, void *data, void *code);
> +FFI_API void *ffi_tramp_get_addr (void *tramp);
> +FFI_API void ffi_tramp_free (void *tramp);
> +
>  /* ---- Public interface definition
> -------------------------------------- */
>
>  FFI_API
> diff --git a/include/ffi_common.h b/include/ffi_common.h
> index 76b9dd6..8743126 100644
> --- a/include/ffi_common.h
> +++ b/include/ffi_common.h
> @@ -103,6 +103,10 @@ ffi_status ffi_prep_cif_core(ffi_cif *cif,
>     some targets.  */
>  void *ffi_data_to_code_pointer (void *data) FFI_HIDDEN;
>
> +/* The arch code calls this to set the code and data parameters for a
> +   closure's static trampoline, if any. */
> +int ffi_closure_tramp_set_parms (void *closure, void *code);
> +
>  /* Extended cif, used in callback from assembly routine */
>  typedef struct
>  {
> diff --git a/libffi.map.in b/libffi.map.in
> index de8778a..049c73e 100644
> --- a/libffi.map.in
> +++ b/libffi.map.in
> @@ -74,3 +74,14 @@ LIBFFI_GO_CLOSURE_8.0 {
>         ffi_prep_go_closure;
>  } LIBFFI_CLOSURE_8.0;
>  #endif
> +
> +#if FFI_EXEC_STATIC_TRAMP
> +LIBFFI_STATIC_TRAMP_8.0 {
> +  global:
> +       ffi_tramp_is_supported;
> +       ffi_tramp_alloc;
> +       ffi_tramp_set_parms;
> +       ffi_tramp_get_addr;
> +       ffi_tramp_free;
> +} LIBFFI_BASE_8.0;
> +#endif
> diff --git a/src/closures.c b/src/closures.c
> index 4fe6158..cf1d3de 100644
> --- a/src/closures.c
> +++ b/src/closures.c
> @@ -109,6 +109,12 @@ ffi_closure_free (void *ptr)
>    munmap(dataseg, rounded_size);
>    munmap(codeseg, rounded_size);
>  }
> +
> +int
> +ffi_closure_tramp_set_parms (void *ptr, void *code)
> +{
> +  return 0;
> +}
>  #else /* !NetBSD with PROT_MPROTECT */
>
>  #if !FFI_MMAP_EXEC_WRIT && !FFI_EXEC_TRAMPOLINE_TABLE
> @@ -843,6 +849,12 @@ dlmmap (void *start, size_t length, int prot,
>           && flags == (MAP_PRIVATE | MAP_ANONYMOUS)
>           && fd == -1 && offset == 0);
>
> +  if (execfd == -1 && ffi_tramp_is_supported ())
> +    {
> +      ptr = mmap (start, length, prot & ~PROT_EXEC, flags, fd, offset);
> +      return ptr;
> +    }
> +
>    if (execfd == -1 && is_emutramp_enabled ())
>      {
>        ptr = mmap (start, length, prot & ~PROT_EXEC, flags, fd, offset);
> @@ -922,7 +934,7 @@ segment_holding_code (mstate m, char* addr)
>  void *
>  ffi_closure_alloc (size_t size, void **code)
>  {
> -  void *ptr;
> +  void *ptr, *ftramp;
>
>    if (!code)
>      return NULL;
> @@ -934,6 +946,17 @@ ffi_closure_alloc (size_t size, void **code)
>        msegmentptr seg = segment_holding (gm, ptr);
>
>        *code = add_segment_exec_offset (ptr, seg);
> +      if (!ffi_tramp_is_supported ())
> +        return ptr;
> +
> +      ftramp = ffi_tramp_alloc (0);
> +      if (ftramp == NULL)
> +      {
> +        dlfree (FFI_RESTORE_PTR (ptr));
> +        return NULL;
> +      }
> +      *code = ffi_tramp_get_addr (ftramp);
> +      ((ffi_closure *) ptr)->ftramp = ftramp;
>      }
>
>    return ptr;
> @@ -943,12 +966,17 @@ void *
>  ffi_data_to_code_pointer (void *data)
>  {
>    msegmentptr seg = segment_holding (gm, data);
> +
>    /* We expect closures to be allocated with ffi_closure_alloc(), in
>       which case seg will be non-NULL.  However, some users take on the
>       burden of managing this memory themselves, in which case this
>       we'll just return data. */
>    if (seg)
> -    return add_segment_exec_offset (data, seg);
> +    {
> +      if (!ffi_tramp_is_supported ())
> +        return add_segment_exec_offset (data, seg);
> +      return ffi_tramp_get_addr (((ffi_closure *) data)->ftramp);
> +    }
>    else
>      return data;
>  }
> @@ -966,10 +994,26 @@ ffi_closure_free (void *ptr)
>    if (seg)
>      ptr = sub_segment_exec_offset (ptr, seg);
>  #endif
> +  if (ffi_tramp_is_supported ())
> +    ffi_tramp_free (((ffi_closure *) ptr)->ftramp);
>
>    dlfree (FFI_RESTORE_PTR (ptr));
>  }
>
> +int
> +ffi_closure_tramp_set_parms (void *ptr, void *code)
> +{
> +  void *ftramp;
> +
> +  msegmentptr seg = segment_holding (gm, ptr);
> +  if (!seg || !ffi_tramp_is_supported())
> +    return 0;
> +
> +  ftramp = ((ffi_closure *) ptr)->ftramp;
> +  ffi_tramp_set_parms (ftramp, code, ptr);
> +  return 1;
> +}
> +
>  # else /* ! FFI_MMAP_EXEC_WRIT */
>
>  /* On many systems, memory returned by malloc is writable and
> @@ -998,6 +1042,12 @@ ffi_data_to_code_pointer (void *data)
>    return data;
>  }
>
> +int
> +ffi_closure_tramp_set_parms (void *ptr, void *code)
> +{
> +  return 0;
> +}
> +
>  # endif /* ! FFI_MMAP_EXEC_WRIT */
>  #endif /* FFI_CLOSURES */
>
> diff --git a/src/tramp.c b/src/tramp.c
> new file mode 100644
> index 0000000..3170152
> --- /dev/null
> +++ b/src/tramp.c
> @@ -0,0 +1,563 @@
> +/* -----------------------------------------------------------------------
> +   tramp.c - Copyright (c) 2020 Madhavan T. Venkataraman
> +
> +   API and support functions for managing statically defined closure
> +   trampolines.
> +
> +   Permission is hereby granted, free of charge, to any person obtaining
> +   a copy of this software and associated documentation files (the
> +   ``Software''), to deal in the Software without restriction, including
> +   without limitation the rights to use, copy, modify, merge, publish,
> +   distribute, sublicense, and/or sell copies of the Software, and to
> +   permit persons to whom the Software is furnished to do so, subject to
> +   the following conditions:
> +
> +   The above copyright notice and this permission notice shall be included
> +   in all copies or substantial portions of the Software.
> +
> +   THE SOFTWARE IS PROVIDED ``AS IS'', WITHOUT WARRANTY OF ANY KIND,
> +   EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
> +   MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
> +   NONINFRINGEMENT.  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
> +   HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
> +   WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
> +   OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
> +   DEALINGS IN THE SOFTWARE.
> +
>  ----------------------------------------------------------------------- */
> +
> +#include <stdio.h>
> +#include <unistd.h>
> +#include <stdlib.h>
> +#include <stdint.h>
> +#include <fcntl.h>
> +#include <pthread.h>
> +#include <sys/mman.h>
> +#include <linux/limits.h>
> +#include <linux/types.h>
> +#include <fficonfig.h>
> +
> +#if defined(FFI_EXEC_STATIC_TRAMP)
> +
> +#if !defined(__linux__)
> +#error "FFI_EXEC_STATIC_TRAMP is currently only supported on Linux"
> +#endif
> +
> +#if !defined _GNU_SOURCE
> +#define _GNU_SOURCE 1
> +#endif
> +
> +/*
> + * Each architecture defines static code for a trampoline code table. The
> + * trampoline code table is mapped into the address space of a process.
> + *
> + * The following architecture specific function returns:
> + *
> + *     - the address of the trampoline code table in the text segment
> + *     - the size of each trampoline in the trampoline code table
> + *     - the size of the mapping for the whole trampoline code table
> + */
> +void __attribute__((weak)) *ffi_tramp_arch (size_t *tramp_size,
> +  size_t *map_size);
> +
> +/* ------------------------- Trampoline Data Structures
> --------------------*/
> +
> +static pthread_mutex_t tramp_lock = PTHREAD_MUTEX_INITIALIZER;
> +
> +struct tramp;
> +
> +/*
> + * Trampoline table. Manages one trampoline code table and one trampoline
> + * parameter table.
> + *
> + * prev, next  Links in the global trampoline table list.
> + * code_table  Trampoline code table mapping.
> + * parm_table  Trampoline parameter table mapping.
> + * array       Array of trampolines malloced.
> + * free                List of free trampolines.
> + * nfree       Number of free trampolines.
> + */
> +struct tramp_table
> +{
> +  struct tramp_table *prev;
> +  struct tramp_table *next;
> +  void *code_table;
> +  void *parm_table;
> +  struct tramp *array;
> +  struct tramp *free;
> +  int nfree;
> +};
> +
> +/*
> + * Parameters for each trampoline.
> + *
> + * data
> + *     Data for the target code that the trampoline jumps to.
> + * target
> + *     Target code that the trampoline jumps to.
> + */
> +struct tramp_parm
> +{
> +  void *data;
> +  void *target;
> +};
> +
> +/*
> + * Trampoline structure for each trampoline.
> + *
> + * prev, next  Links in the trampoline free list of a trampoline table.
> + * table       Trampoline table to which this trampoline belongs.
> + * code                Address of this trampoline in the code table
> mapping.
> + * parm                Address of this trampoline's parameters in the
> parameter
> + *             table mapping.
> + */
> +struct tramp
> +{
> +  struct tramp *prev;
> +  struct tramp *next;
> +  struct tramp_table *table;
> +  void *code;
> +  struct tramp_parm *parm;
> +};
> +
> +/*
> + * Trampoline globals.
> + *
> + * fd
> + *     File descriptor of binary file that contains the trampoline code
> table.
> + * offset
> + *     Offset of the trampoline code table in that file.
> + * map_size
> + *     Size of the trampoline code table mapping.
> + * size
> + *     Size of one trampoline in the trampoline code table.
> + * ntramp
> + *     Total number of trampolines in the trampoline code table.
> + * tables
> + *     List of trampoline tables that contain free trampolines.
> + * ntables
> + *     Number of trampoline tables that contain free trampolines.
> + */
> +struct tramp_global
> +{
> +  int fd;
> +  off_t offset;
> +  size_t map_size;
> +  size_t size;
> +  int ntramp;
> +  struct tramp_table *tables;
> +  int ntables;
> +};
> +
> +static struct tramp_global gtramp = { -1 };
> +
> +/* ------------------------ Trampoline Initialization
> ----------------------*/
> +
> +static int ffi_tramp_get_fd_offset (void *tramp_text, off_t *offset);
> +
> +/*
> + * Initialize the static trampoline feature.
> + */
> +static int
> +ffi_tramp_init (void)
> +{
> +  if (ffi_tramp_arch == NULL)
> +    return 0;
> +
> +  if (gtramp.fd == -1)
> +    {
> +      void *tramp_text;
> +
> +      gtramp.tables = NULL;
> +      gtramp.ntables = 0;
> +
> +      /*
> +       * Get trampoline code table information from the architecture.
> +       */
> +      tramp_text = ffi_tramp_arch (&gtramp.size, &gtramp.map_size);
> +      gtramp.ntramp = gtramp.map_size / gtramp.size;
> +
> +      /*
> +       * Get the binary file that contains the trampoline code table and
> also
> +       * the offset of the table within the file. These are used to mmap()
> +       * the trampoline code table.
> +       */
> +      gtramp.fd = ffi_tramp_get_fd_offset (tramp_text, &gtramp.offset);
> +    }
> +  return gtramp.fd != -1;
> +}
> +
> +/*
> + * From the address of the trampoline code table in the text segment,
> find the
> + * binary file and offset of the trampoline code table from
> /proc/<pid>/maps.
> + */
> +static int
> +ffi_tramp_get_fd_offset (void *tramp_text, off_t *offset)
> +{
> +  FILE *fp;
> +  char file[PATH_MAX], line[PATH_MAX+100], perm[10], dev[10];
> +  unsigned long start, end, inode;
> +  uintptr_t addr = (uintptr_t) tramp_text;
> +  int nfields, found;
> +
> +  snprintf (file, PATH_MAX, "/proc/%d/maps", getpid());
> +  fp = fopen (file, "r");
> +  if (fp == NULL)
> +    return -1;
> +
> +  found = 0;
> +  while (feof (fp) == 0) {
> +    if (fgets (line, sizeof (line), fp) == 0)
> +      break;
> +
> +    nfields = sscanf (line, "%lx-%lx %s %lx %s %ld %s",
> +      &start, &end, perm, offset, dev, &inode, file);
> +    if (nfields != 7)
> +      continue;
> +
> +    if (addr >= start && addr < end) {
> +      *offset += (addr - start);
> +      found = 1;
> +      break;
> +    }
> +  }
> +  fclose (fp);
> +
> +  if (!found)
> +    return -1;
> +
> +  return open (file, O_RDONLY);
> +}
> +
> +/* ---------------------- Trampoline Table functions
> ---------------------- */
> +
> +static int tramp_table_map (char **code_table, char **parm_table);
> +static void tramp_add (struct tramp *tramp);
> +
> +/*
> + * Allocate and initialize a trampoline table.
> + */
> +static int
> +tramp_table_alloc (void)
> +{
> +  char *code_table, *parm_table;
> +  struct tramp_table *table;
> +  struct tramp *tramp_array, *tramp;
> +  size_t size;
> +  char *code, *parm;
> +  int i;
> +
> +  /*
> +   * If we already have tables with free trampolines, there is no need to
> +   * allocate a new table.
> +   */
> +  if (gtramp.ntables > 0)
> +    return 1;
> +
> +  /*
> +   * Allocate a new trampoline table structure.
> +   */
> +  table = malloc (sizeof (*table));
> +  if (table == NULL)
> +    return 0;
> +
> +  /*
> +   * Allocate new trampoline structures.
> +   */
> +  tramp_array = malloc (sizeof (*tramp) * gtramp.ntramp);
> +  if (tramp_array == NULL)
> +    goto free_table;
> +
> +  /*
> +   * Map a code table and a parameter table into the caller's address
> space.
> +   */
> +  if (!tramp_table_map (&code_table, &parm_table))
> +    goto free_tramp_array;
> +
> +  /*
> +   * Initialize the trampoline table.
> +   */
> +  table->code_table = code_table;
> +  table->parm_table = parm_table;
> +  table->array = tramp_array;
> +  table->free = NULL;
> +  table->nfree = 0;
> +
> +  /*
> +   * Populate the trampoline table free list. This will also add the
> trampoline
> +   * table to the global list of trampoline tables.
> +   */
> +  size = gtramp.size;
> +  code = code_table;
> +  parm = parm_table;
> +  for (i = 0; i < gtramp.ntramp; i++)
> +    {
> +      tramp = &tramp_array[i];
> +      tramp->table = table;
> +      tramp->code = code;
> +      tramp->parm = (struct tramp_parm *) parm;
> +      tramp_add (tramp);
> +
> +      code += size;
> +      parm += size;
> +    }
> +  return 1;
> +
> +free_tramp_array:
> +  free (tramp_array);
> +free_table:
> +  free (table);
> +  return 0;
> +}
> +
> +/*
> + * Create a trampoline code table mapping and a trampoline parameter table
> + * mapping. The two mappings must be adjacent to each other for
> PC-relative
> + * access.
> + *
> + * For each trampoline in the code table, there is a corresponding
> parameter
> + * block in the parameter table. The size of the parameter block is the
> same
> + * as the size of the trampoline. This means that the parameter block is
> at
> + * a fixed offset from its trampoline making it easy for a trampoline to
> find
> + * its parameters using PC-relative access.
> + *
> + * The parameter block will contain a struct tramp_parm. This means that
> + * sizeof (struct tramp_parm) cannot exceed the size of a parameter block.
> + */
> +static int
> +tramp_table_map (char **code_table, char **parm_table)
> +{
> +  char *addr;
> +
> +  /*
> +   * Create an anonymous mapping twice the map size. The top half will be
> used
> +   * for the code table. The bottom half will be used for the parameter
> table.
> +   */
> +  addr = mmap (NULL, gtramp.map_size * 2, PROT_READ | PROT_WRITE,
> +    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
> +  if (addr == MAP_FAILED)
> +    return 0;
> +
> +  /*
> +   * Replace the top half of the anonymous mapping with the code table
> mapping.
> +   */
> +  *code_table = mmap (addr, gtramp.map_size, PROT_READ | PROT_EXEC,
> +    MAP_PRIVATE | MAP_FIXED, gtramp.fd, gtramp.offset);
> +  if (*code_table == MAP_FAILED)
> +    {
> +      (void) munmap (addr, gtramp.map_size * 2);
> +      return 0;
> +    }
> +  *parm_table = *code_table + gtramp.map_size;
> +  return 1;
> +}
> +
> +/*
> + * Free a trampoline table.
> + */
> +static void
> +tramp_table_free (struct tramp_table *table)
> +{
> +  (void) munmap (table->code_table, gtramp.map_size);
> +  (void) munmap (table->parm_table, gtramp.map_size);
> +  free (table->array);
> +  free (table);
> +}
> +
> +/*
> + * Add a new trampoline table to the global table list.
> + */
> +static void
> +tramp_table_add (struct tramp_table *table)
> +{
> +  table->next = gtramp.tables;
> +  table->prev = NULL;
> +  if (gtramp.tables != NULL)
> +    gtramp.tables->prev = table;
> +  gtramp.tables = table;
> +  gtramp.ntables++;
> +}
> +
> +/*
> + * Delete a trampoline table from the global table list.
> + */
> +static void
> +tramp_table_del (struct tramp_table *table)
> +{
> +  gtramp.ntables--;
> +  if (table->prev != NULL)
> +    table->prev->next = table->next;
> +  if (table->next != NULL)
> +    table->next->prev = table->prev;
> +  if (gtramp.tables == table)
> +    gtramp.tables = table->next;
> +}
> +
> +/* ------------------------- Trampoline functions
> ------------------------- */
> +
> +/*
> + * Add a trampoline to its trampoline table.
> + */
> +static void
> +tramp_add (struct tramp *tramp)
> +{
> +  struct tramp_table *table = tramp->table;
> +
> +  tramp->next = table->free;
> +  tramp->prev = NULL;
> +  if (table->free != NULL)
> +    table->free->prev = tramp;
> +  table->free = tramp;
> +  table->nfree++;
> +
> +  if (table->nfree == 1)
> +    tramp_table_add (table);
> +
> +  /*
> +   * We don't want to keep too many free trampoline tables lying around.
> +   */
> +  if (table->nfree == gtramp.ntramp && gtramp.ntables > 1)
> +    {
> +      tramp_table_del (table);
> +      tramp_table_free (table);
> +    }
> +}
> +
> +/*
> + * Remove a trampoline from its trampoline table.
> + */
> +static void
> +tramp_del (struct tramp *tramp)
> +{
> +  struct tramp_table *table = tramp->table;
> +
> +  table->nfree--;
> +  if (tramp->prev != NULL)
> +    tramp->prev->next = tramp->next;
> +  if (tramp->next != NULL)
> +    tramp->next->prev = tramp->prev;
> +  if (table->free == tramp)
> +    table->free = tramp->next;
> +
> +  if (table->nfree == 0)
> +    tramp_table_del (table);
> +}
> +
> +/* ------------------------ Trampoline API functions
> ------------------------ */
> +
> +int
> +ffi_tramp_is_supported(void)
> +{
> +  int ret;
> +
> +  pthread_mutex_lock (&tramp_lock);
> +  ret = ffi_tramp_init ();
> +  pthread_mutex_unlock (&tramp_lock);
> +  return ret;
> +}
> +
> +/*
> + * Allocate a trampoline and return its opaque address.
> + */
> +void *
> +ffi_tramp_alloc (int flags)
> +{
> +  struct tramp *tramp;
> +
> +  pthread_mutex_lock (&tramp_lock);
> +
> +  if (!ffi_tramp_init () || flags != 0)
> +    {
> +      pthread_mutex_unlock (&tramp_lock);
> +      return NULL;
> +    }
> +
> +  if (!tramp_table_alloc ())
> +    {
> +      pthread_mutex_unlock (&tramp_lock);
> +      return NULL;
> +    }
> +
> +  tramp = gtramp.tables->free;
> +  tramp_del (tramp);
> +
> +  pthread_mutex_unlock (&tramp_lock);
> +
> +  return tramp;
> +}
> +
> +/*
> + * Set the parameters for a trampoline.
> + */
> +void
> +ffi_tramp_set_parms (void *arg, void *target, void *data)
> +{
> +  struct tramp *tramp = arg;
> +
> +  pthread_mutex_lock (&tramp_lock);
> +  tramp->parm->target = target;
> +  tramp->parm->data = data;
> +  pthread_mutex_unlock (&tramp_lock);
> +}
> +
> +/*
> + * Get the invocation address of a trampoline.
> + */
> +void *
> +ffi_tramp_get_addr (void *arg)
> +{
> +  struct tramp *tramp = arg;
> +  void *addr;
> +
> +  pthread_mutex_lock (&tramp_lock);
> +  addr = tramp->code;
> +  pthread_mutex_unlock (&tramp_lock);
> +
> +  return addr;
> +}
> +
> +/*
> + * Free a trampoline.
> + */
> +void
> +ffi_tramp_free (void *arg)
> +{
> +  struct tramp *tramp = arg;
> +
> +  pthread_mutex_lock (&tramp_lock);
> +  tramp_add (tramp);
> +  pthread_mutex_unlock (&tramp_lock);
> +}
> +
> +/*
> ------------------------------------------------------------------------- */
> +
> +#else /* !FFI_EXEC_STATIC_TRAMP */
> +
> +int
> +ffi_tramp_is_supported(void)
> +{
> +  return 0;
> +}
> +
> +void *
> +ffi_tramp_alloc (int flags)
> +{
> +  return NULL;
> +}
> +
> +void
> +ffi_tramp_set_parms (void *arg, void *target, void *data)
> +{
> +}
> +
> +void *
> +ffi_tramp_get_addr (void *arg)
> +{
> +  return NULL;
> +}
> +
> +void
> +ffi_tramp_free (void *arg)
> +{
> +}
> +
> +#endif /* FFI_EXEC_STATIC_TRAMP */
> diff --git a/testsuite/libffi.closures/closure_loc_fn0.c
> b/testsuite/libffi.closures/closure_loc_fn0.c
> index b3afa0b..ad488ac 100644
> --- a/testsuite/libffi.closures/closure_loc_fn0.c
> +++ b/testsuite/libffi.closures/closure_loc_fn0.c
> @@ -83,7 +83,10 @@ int main (void)
>    CHECK(ffi_prep_closure_loc(pcl, &cif, closure_loc_test_fn0,
>                          (void *) 3 /* userdata */, codeloc) == FFI_OK);
>
> +#ifndef FFI_EXEC_STATIC_TRAMP
> +  /* With static trampolines, the codeloc does not point to closure */
>    CHECK(memcmp(pcl, codeloc, sizeof(*pcl)) == 0);
> +#endif
>
>    res = (*((closure_loc_test_type0)codeloc))
>      (1LL, 2, 3LL, 4, 127, 429LL, 7, 8, 9.5, 10, 11, 12, 13,
> --
> 2.25.1
>
>