* [PATCHv6 0/4] glibc tunables
@ 2016-11-16 8:35 Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 4/4] User manual documentation for tunables Siddhesh Poyarekar
` (10 more replies)
0 siblings, 11 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-16 8:35 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Hi,
Here is iteration 6 of the tunables patchset, now with two patches palmed off
to Adhemerval to worry about! Here are changes from the previous patchset:
- I discovered some implicit includes that broke aarch64 - shame on me for
missing them.
- I realized that __tunables_init used strncmp and strlen. They're built into
ld.so too, but they are likely tunables targets and I don't want them to
break unintentionally in future, so off they go
- The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
my comments on the patch.
The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
patch. If you're watching this, please also review Adhemerval's
__access_noerrno patch and help get that through.
Siddhesh
[1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
Siddhesh Poyarekar (4):
Add framework for tunables
Initialize tunable list with the GLIBC_TUNABLES environment variable
Enhance --enable-tunables to select tunables frontend at build time
User manual documentation for tunables
INSTALL | 18 ++
Makeconfig | 16 ++
README.tunables | 84 ++++++
config.h.in | 3 +
config.make.in | 1 +
configure | 17 ++
configure.ac | 10 +
csu/init-first.c | 2 -
csu/libc-start.c | 12 +
elf/Makefile | 7 +
elf/Versions | 3 +
elf/dl-support.c | 2 +
elf/dl-sysdep.c | 8 +
elf/dl-tunable-types.h | 46 +++
elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
elf/dl-tunables.h | 82 ++++++
elf/dl-tunables.list | 69 +++++
elf/rtld.c | 2 +
malloc/Makefile | 6 +
malloc/arena.c | 54 ++++
malloc/tst-malloc-usable-static-tunables.c | 1 +
malloc/tst-malloc-usable-static.c | 1 +
malloc/tst-malloc-usable-tunables.c | 1 +
manual/Makefile | 3 +-
manual/install.texi | 21 ++
manual/probes.texi | 2 +-
manual/tunables.texi | 185 ++++++++++++
scripts/gen-tunables.awk | 157 ++++++++++
sysdeps/mach/hurd/dl-sysdep.c | 8 +
29 files changed, 1264 insertions(+), 4 deletions(-)
create mode 100644 README.tunables
create mode 100644 elf/dl-tunable-types.h
create mode 100644 elf/dl-tunables.c
create mode 100644 elf/dl-tunables.h
create mode 100644 elf/dl-tunables.list
create mode 100644 malloc/tst-malloc-usable-static-tunables.c
create mode 100644 malloc/tst-malloc-usable-static.c
create mode 100644 malloc/tst-malloc-usable-tunables.c
create mode 100644 manual/tunables.texi
create mode 100644 scripts/gen-tunables.awk
--
2.7.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 1/4] Add framework for tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 4/4] User manual documentation for tunables Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable Siddhesh Poyarekar
@ 2016-11-16 8:35 ` Siddhesh Poyarekar
2016-12-27 11:08 ` Florian Weimer
2016-11-16 8:35 ` [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time Siddhesh Poyarekar
` (7 subsequent siblings)
10 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-16 8:35 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
The tunables framework allows us to uniformly manage and expose global
variables inside glibc as switches to users. tunables/README has
instructions for glibc developers to add new tunables.
Tunables support can be enabled by passing the --enable-tunables
configure flag to the configure script. This patch only adds a
framework and does not pose any limitations on how tunable values are
read from the user. It also adds environment variables used in malloc
behaviour tweaking to the tunables framework as a PoC of the
compatibility interface.
* manual/install.texi: Add --enable-tunables option.
* INSTALL: Regenerate.
* README.tunables: New file.
* Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
(before-compile): Generate dl-tunable-list.h early.
* config.h.in: Add HAVE_TUNABLES.
* config.make.in: Add have-tunables.
* configure.ac: Add --enable-tunables option.
* configure: Regenerate.
* csu/init-first.c (__libc_init_first): Move
__libc_init_secure earlier...
* csu/libc-start.c (LIBC_START_MAIN):... to here.
[HAVE_TUNABLES]: Include dl-tunables.h.
(LIBC_START_MAIN) [!SHARED]: Initialize tunables for static
binaries.
* elf/Makefile (dl-routines): Add dl-tunables.
* elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE
namespace.
* elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_
only when !HAVE_TUNABLES.
* elf/rtld.c (process_envvars): Likewise.
* elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h
(_dl_sysdep_start): Call __tunables_init.
* elf/dl-tunable-types.h: New file.
* elf/dl-tunables.c: New file.
* elf/dl-tunables.h: New file.
* elf/dl-tunables.list: New file.
* malloc/tst-malloc-usable-static.c: New test case.
* malloc/Makefile (tests-static): Add it.
* malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h.
Define TUNABLE_NAMESPACE.
(DL_TUNABLE_CALLBACK (set_mallopt_check)): New function.
(DL_TUNABLE_CALLBACK_FNDECL): New macro. Use it to define
callback functions.
(ptmalloc_init): Set tunable values.
* scripts/gen-tunables.awk: New file.
* sysdeps/mach/hurd/dl-sysdep.c [HAVE_TUNABLES]: Include
dl-tunables.h.
(_dl_sysdep_start): Call __tunables_init.
---
INSTALL | 5 +
Makeconfig | 16 ++
README.tunables | 84 +++++++++++
config.h.in | 3 +
config.make.in | 1 +
configure | 16 ++
configure.ac | 10 ++
csu/init-first.c | 2 -
csu/libc-start.c | 12 ++
elf/Makefile | 5 +
elf/Versions | 3 +
elf/dl-support.c | 2 +
elf/dl-sysdep.c | 8 +
elf/dl-tunable-types.h | 46 ++++++
elf/dl-tunables.c | 310 ++++++++++++++++++++++++++++++++++++++
elf/dl-tunables.h | 78 ++++++++++
elf/dl-tunables.list | 69 +++++++++
elf/rtld.c | 2 +
malloc/Makefile | 2 +
malloc/arena.c | 54 +++++++
malloc/tst-malloc-usable-static.c | 1 +
manual/install.texi | 5 +
scripts/gen-tunables.awk | 157 +++++++++++++++++++
sysdeps/mach/hurd/dl-sysdep.c | 8 +
24 files changed, 897 insertions(+), 2 deletions(-)
create mode 100644 README.tunables
create mode 100644 elf/dl-tunable-types.h
create mode 100644 elf/dl-tunables.c
create mode 100644 elf/dl-tunables.h
create mode 100644 elf/dl-tunables.list
create mode 100644 malloc/tst-malloc-usable-static.c
create mode 100644 scripts/gen-tunables.awk
diff --git a/INSTALL b/INSTALL
index b5acedc..ccacccc 100644
--- a/INSTALL
+++ b/INSTALL
@@ -158,6 +158,11 @@ will be used, and CFLAGS sets optimization options for the compiler.
By default for x86_64, the GNU C Library is built with the vector
math library. Use this option to disable the vector math library.
+'--enable-tunables'
+ Tunables support allows additional library parameters to be
+ customized at runtime. This is an experimental feature and affects
+ startup time and is thus disabled by default.
+
'--build=BUILD-SYSTEM'
'--host=HOST-SYSTEM'
These options are for cross-compiling. If you specify both options
diff --git a/Makeconfig b/Makeconfig
index a785860..d0eb832 100644
--- a/Makeconfig
+++ b/Makeconfig
@@ -883,6 +883,11 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \
$(foreach lib,$(libof-$(basename $(@F))) \
$(libof-$(<F)) $(libof-$(@F)),$(CPPFLAGS-$(lib))) \
$(CPPFLAGS-$(<F)) $(CPPFLAGS-$(@F)) $(CPPFLAGS-$(basename $(@F)))
+
+ifeq (yes,$(have-tunables))
+CPPFLAGS += -DTOP_NAMESPACE=glibc
+endif
+
override CFLAGS = -std=gnu11 -fgnu89-inline $(config-extra-cflags) \
$(filter-out %frame-pointer,$(+cflags)) $(+gccwarn-c) \
$(sysdep-CFLAGS) $(CFLAGS-$(suffix $@)) $(CFLAGS-$(<F)) \
@@ -1057,6 +1062,17 @@ $(common-objpfx)libc-modules.stmp: $(..)scripts/gen-libc-modules.awk \
endif
+# Build the tunables list header early since it could be used by any module in
+# glibc.
+ifeq (yes,$(have-tunables))
+before-compile += $(common-objpfx)dl-tunable-list.h
+
+$(common-objpfx)dl-tunable-list.h: $(..)scripts/gen-tunables.awk \
+ $(..)elf/dl-tunables.list
+ $(AWK) -f $^ > $@.tmp
+ mv $@.tmp $@
+endif
+
common-generated += libc-modules.h libc-modules.stmp
# The name under which the run-time dynamic linker is installed.
diff --git a/README.tunables b/README.tunables
new file mode 100644
index 0000000..3d9c86a
--- /dev/null
+++ b/README.tunables
@@ -0,0 +1,84 @@
+ TUNABLE FRAMEWORK
+ =================
+
+Tunables is a feature in the GNU C Library that allows application authors and
+distribution maintainers to alter the runtime library behaviour to match their
+workload.
+
+The tunable framework allows modules within glibc to register variables that
+may be tweaked through an environment variable or an API call. It aims to
+enforce a strict namespace rule to bring consistency to naming of these tunable
+environment variables across the project. This document is a guide for glibc
+developers to add tunables to the framework.
+
+ADDING A NEW TUNABLE
+--------------------
+
+The TOP_NAMESPACE is defined by default as 'glibc' and it may be overridden in
+distributions for specific tunables if they want to add their own tunables.
+Downstream implementations are discouraged from using the 'glibc' namespace for
+tunables they don't already have consensus to push upstream.
+
+There are two steps to adding a tunable:
+
+1. Add a tunable ID:
+
+Modules that wish to use the tunables interface must define the
+TUNABLE_NAMESPACE macro. Following this, for each tunable you want to
+add, make an entry in elf/dl-tunables.list. The format of the file is as
+follows:
+
+TOP_NAMESPACE {
+ NAMESPACE1 {
+ TUNABLE1 {
+ # tunable attributes, one per line
+ }
+ # A tunable with default attributes, i.e. string variable.
+ TUNABLE2
+ TUNABLE3 {
+ # its attributes
+ }
+ }
+ NAMESPACE2 {
+ ...
+ }
+}
+
+The list of allowed attributes are:
+
+- type: Data type. Defaults to STRING. Allowed types are:
+ INT_32, SIZE_T and STRING.
+
+- minval: Optional minimum acceptable value. For a string type
+ this is the minimum length of the value.
+
+- maxval: Optional maximum acceptable value. For a string type
+ this is the maximum length of the value.
+
+- env_alias: An alias environment variable
+
+- is_secure: Specify whether the tunable should be read for setuid
+ binaries. True allows the tunable to be read for
+ setuid binaries while false disables it. Note that
+ even if this is set as true and the value is read, it
+ may not be used if it does not validate against the
+ acceptable values or is not considered safe by the
+ module.
+
+2. Call either the TUNABLE_SET_VALUE and pass into it the tunable name and a
+ pointer to the variable that should be set with the tunable value.
+ If additional work needs to be done after setting the value, use the
+ TUNABLE_SET_VALUE_WITH_CALLBACK instead and additionally pass a pointer to
+ the function that should be called if the tunable value has been set.
+
+FUTURE WORK
+-----------
+
+The framework currently only allows a one-time initialization of variables
+through environment variables and in some cases, modification of variables via
+an API call. A future goals for this project include:
+
+- Setting system-wide and user-wide defaults for tunables through some
+ mechanism like a configuration file.
+
+- Allow tweaking of some tunables at runtime
diff --git a/config.h.in b/config.h.in
index 33757bd..fd93bb3 100644
--- a/config.h.in
+++ b/config.h.in
@@ -246,4 +246,7 @@
/* PowerPC32 uses fctidz for floating point to long long conversions. */
#define HAVE_PPC_FCTIDZ 0
+/* Build glibc with tunables support. */
+#define HAVE_TUNABLES 0
+
#endif
diff --git a/config.make.in b/config.make.in
index 04a8b3e..6d0b607 100644
--- a/config.make.in
+++ b/config.make.in
@@ -93,6 +93,7 @@ use-nscd = @use_nscd@
build-hardcoded-path-in-tests= @hardcoded_path_in_tests@
build-pt-chown = @build_pt_chown@
enable-lock-elision = @enable_lock_elision@
+have-tunables = @have_tunables@
# Build tools.
CC = @CC@
diff --git a/configure b/configure
index d9f8c06..cbd5fb3 100755
--- a/configure
+++ b/configure
@@ -662,6 +662,7 @@ multi_arch
base_machine
add_on_subdirs
add_ons
+have_tunables
build_pt_chown
build_nscd
link_obsolete_rpc
@@ -776,6 +777,7 @@ enable_systemtap
enable_build_nscd
enable_nscd
enable_pt_chown
+enable_tunables
enable_mathvec
with_cpu
'
@@ -1443,6 +1445,7 @@ Optional Features:
--disable-build-nscd disable building and installing the nscd daemon
--disable-nscd library functions will not contact the nscd daemon
--enable-pt_chown Enable building and installing pt_chown
+ --enable-tunables Enable tunables support
--enable-mathvec Enable building and installing mathvec [default
depends on architecture]
@@ -3649,6 +3652,19 @@ if test "$build_pt_chown" = yes; then
fi
+# Check whether --enable-tunables was given.
+if test "${enable_tunables+set}" = set; then :
+ enableval=$enable_tunables; have_tunables=$enableval
+else
+ have_tunables=no
+fi
+
+
+if test "$have_tunables" = yes; then
+ $as_echo "#define HAVE_TUNABLES 1" >>confdefs.h
+
+fi
+
# The abi-tags file uses a fairly simplistic model for name recognition that
# can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
# $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
diff --git a/configure.ac b/configure.ac
index de0a40f..3b7a0db 100644
--- a/configure.ac
+++ b/configure.ac
@@ -395,6 +395,16 @@ if test "$build_pt_chown" = yes; then
AC_DEFINE(HAVE_PT_CHOWN)
fi
+AC_ARG_ENABLE([tunables],
+ [AS_HELP_STRING([--enable-tunables],
+ [Enable tunables support])],
+ [have_tunables=$enableval],
+ [have_tunables=no])
+AC_SUBST(have_tunables)
+if test "$have_tunables" = yes; then
+ AC_DEFINE(HAVE_TUNABLES)
+fi
+
# The abi-tags file uses a fairly simplistic model for name recognition that
# can't distinguish i486-pc-linux-gnu fully from i486-pc-gnu. So we mutate a
# $host_os of `gnu*' here to be `gnu-gnu*' just so that it can tell.
diff --git a/csu/init-first.c b/csu/init-first.c
index 77c6e1c..465f25b 100644
--- a/csu/init-first.c
+++ b/csu/init-first.c
@@ -72,8 +72,6 @@ _init (int argc, char **argv, char **envp)
__environ = envp;
#ifndef SHARED
- __libc_init_secure ();
-
/* First the initialization which normally would be done by the
dynamic linker. */
_dl_non_dynamic_init ();
diff --git a/csu/libc-start.c b/csu/libc-start.c
index 99c040a..4e5cafa 100644
--- a/csu/libc-start.c
+++ b/csu/libc-start.c
@@ -21,6 +21,11 @@
#include <unistd.h>
#include <ldsodefs.h>
#include <exit-thread.h>
+#include <libc-internal.h>
+
+#if HAVE_TUNABLES
+# include <elf/dl-tunables.h>
+#endif
extern void __libc_init_first (int argc, char **argv, char **envp);
@@ -184,6 +189,13 @@ LIBC_START_MAIN (int (*main) (int, char **, char ** MAIN_AUXVEC_DECL),
}
# endif
+ /* Initialize very early so that tunables can use it. */
+ __libc_init_secure ();
+
+#if HAVE_TUNABLES
+ __tunables_init (__environ);
+#endif
+
/* Perform IREL{,A} relocations. */
apply_irel ();
diff --git a/elf/Makefile b/elf/Makefile
index 82c7e05..477f8fc 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -35,6 +35,11 @@ dl-routines = $(addprefix dl-,load lookup object reloc deps hwcaps \
ifeq (yes,$(use-ldconfig))
dl-routines += dl-cache
endif
+
+ifeq (yes,$(have-tunables))
+dl-routines += dl-tunables
+endif
+
all-dl-routines = $(dl-routines) $(sysdep-dl-routines)
# But they are absent from the shared libc, because that code is in ld.so.
elide-routines.os = $(all-dl-routines) dl-support enbl-secure dl-origin \
diff --git a/elf/Versions b/elf/Versions
index 23deda9..1734697 100644
--- a/elf/Versions
+++ b/elf/Versions
@@ -64,5 +64,8 @@ ld {
# Pointer protection.
__pointer_chk_guard;
+
+ # Set value of a tunable.
+ __tunable_set_val;
}
}
diff --git a/elf/dl-support.c b/elf/dl-support.c
index c30194c..d350d6d 100644
--- a/elf/dl-support.c
+++ b/elf/dl-support.c
@@ -354,8 +354,10 @@ _dl_non_dynamic_init (void)
cp = (const char *) __rawmemchr (cp, '\0') + 1;
}
+#if !HAVE_TUNABLES
if (__access ("/etc/suid-debug", F_OK) != 0)
__unsetenv ("MALLOC_CHECK_");
+#endif
}
#ifdef DL_PLATFORM_INIT
diff --git a/elf/dl-sysdep.c b/elf/dl-sysdep.c
index eaa7155..68f30f2 100644
--- a/elf/dl-sysdep.c
+++ b/elf/dl-sysdep.c
@@ -44,6 +44,10 @@
#include <hp-timing.h>
#include <tls.h>
+#if HAVE_TUNABLES
+# include <dl-tunables.h>
+#endif
+
extern char **_environ attribute_hidden;
extern char _end[] attribute_hidden;
@@ -219,6 +223,10 @@ _dl_sysdep_start (void **start_argptr,
}
#endif
+#if HAVE_TUNABLES
+ __tunables_init (_environ);
+#endif
+
#ifdef DL_SYSDEP_INIT
DL_SYSDEP_INIT;
#endif
diff --git a/elf/dl-tunable-types.h b/elf/dl-tunable-types.h
new file mode 100644
index 0000000..d1591b6
--- /dev/null
+++ b/elf/dl-tunable-types.h
@@ -0,0 +1,46 @@
+/* Tunable type information.
+
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _TUNABLE_TYPES_H_
+# define _TUNABLE_TYPES_H_
+#include <stddef.h>
+
+typedef void (*tunable_callback_t) (void *);
+
+typedef enum
+{
+ TUNABLE_TYPE_INT_32,
+ TUNABLE_TYPE_SIZE_T,
+ TUNABLE_TYPE_STRING
+} tunable_type_code_t;
+
+typedef struct
+{
+ tunable_type_code_t type_code;
+ int64_t min;
+ int64_t max;
+} tunable_type_t;
+
+typedef union
+{
+ int64_t numval;
+ const char *strval;
+} tunable_val_t;
+
+#endif
diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
new file mode 100644
index 0000000..91340b6
--- /dev/null
+++ b/elf/dl-tunables.c
@@ -0,0 +1,310 @@
+/* The tunable framework. See the README to know how to use the tunable in
+ a glibc module.
+
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#include <stdint.h>
+#include <stdbool.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <libc-internal.h>
+#include <sysdep.h>
+#include <fcntl.h>
+
+#define TUNABLES_INTERNAL 1
+#include "dl-tunables.h"
+
+/* Compare environment names, bounded by the name hardcoded in glibc. */
+static bool
+is_name (const char *orig, const char *envname)
+{
+ for (;*orig != '\0' && *envname != '\0'; envname++, orig++)
+ if (*orig != *envname)
+ break;
+
+ /* The ENVNAME is immediately followed by a value. */
+ if (*orig == '\0' && *envname == '=')
+ return true;
+ else
+ return false;
+}
+
+static char **
+get_next_env (char **envp, char **name, size_t *namelen, char **val)
+{
+ while (envp != NULL && *envp != NULL)
+ {
+ char *envline = *envp;
+ int len = 0;
+
+ while (envline[len] != '\0' && envline[len] != '=')
+ len++;
+
+ /* Just the name and no value, go to the next one. */
+ if (envline[len] == '\0')
+ continue;
+
+ *name = envline;
+ *namelen = len;
+ *val = &envline[len + 1];
+
+ return ++envp;
+ }
+
+ return NULL;
+}
+
+static int
+tunables_unsetenv (char **ep, const char *name)
+{
+ while (*ep != NULL)
+ {
+ size_t cnt = 0;
+
+ while ((*ep)[cnt] == name[cnt] && name[cnt] != '\0')
+ ++cnt;
+
+ if (name[cnt] == '\0' && (*ep)[cnt] == '=')
+ {
+ /* Found it. Remove this pointer by moving later ones to
+ the front. */
+ char **dp = ep;
+
+ do
+ dp[0] = dp[1];
+ while (*dp++);
+ /* Continue the loop in case NAME appears again. */
+ }
+ else
+ ++ep;
+ }
+
+ return 0;
+}
+
+/* A stripped down strtoul-like implementation for very early use. It does not
+ set errno if the result is outside bounds because it gets called before
+ errno may have been set up. */
+static unsigned long int
+tunables_strtoul (const char *nptr)
+{
+ unsigned long int result = 0;
+ long int sign = 1;
+ unsigned max_digit;
+
+ while (*nptr == ' ' || *nptr == '\t')
+ ++nptr;
+
+ if (*nptr == '-')
+ {
+ sign = -1;
+ ++nptr;
+ }
+ else if (*nptr == '+')
+ ++nptr;
+
+ if (*nptr < '0' || *nptr > '9')
+ return 0UL;
+
+ int base = 10;
+ max_digit = 9;
+ if (*nptr == '0')
+ {
+ if (nptr[1] == 'x' || nptr[1] == 'X')
+ {
+ base = 16;
+ nptr += 2;
+ }
+ else
+ {
+ base = 8;
+ max_digit = 7;
+ }
+ }
+
+ while (1)
+ {
+ unsigned long int digval;
+ if (*nptr >= '0' && *nptr <= '0' + max_digit)
+ digval = *nptr - '0';
+ else if (base == 16)
+ {
+ if (*nptr >= 'a' && *nptr <= 'f')
+ digval = *nptr - 'a' + 10;
+ else if (*nptr >= 'A' && *nptr <= 'F')
+ digval = *nptr - 'A' + 10;
+ else
+ break;
+ }
+ else
+ break;
+
+ if (result > ULONG_MAX / base
+ || (result == ULONG_MAX / base && digval > ULONG_MAX % base))
+ return ULONG_MAX;
+ result *= base;
+ result += digval;
+ ++nptr;
+ }
+
+ return result * sign;
+}
+
+/* If the value does not validate, don't bother initializing the internal type
+ and also take care to clear the recorded string value in STRVAL. */
+#define RETURN_IF_INVALID_RANGE(__cur, __val) \
+({ \
+ __typeof ((__cur)->type.min) min = (__cur)->type.min; \
+ __typeof ((__cur)->type.max) max = (__cur)->type.max; \
+ if (min != max && ((__val) < min || (__val) > max)) \
+ return; \
+})
+
+/* Validate range of the input value and initialize the tunable CUR if it looks
+ good. */
+static void
+tunable_initialize (tunable_t *cur, const char *strval)
+{
+ switch (cur->type.type_code)
+ {
+ case TUNABLE_TYPE_INT_32:
+ {
+ int32_t val = (int32_t) tunables_strtoul (strval);
+ RETURN_IF_INVALID_RANGE (cur, val);
+ cur->val.numval = (int64_t) val;
+ cur->strval = strval;
+ break;
+ }
+ case TUNABLE_TYPE_SIZE_T:
+ {
+ size_t val = (size_t) tunables_strtoul (strval);
+ RETURN_IF_INVALID_RANGE (cur, val);
+ cur->val.numval = (int64_t) val;
+ cur->strval = strval;
+ break;
+ }
+ case TUNABLE_TYPE_STRING:
+ {
+ cur->val.strval = cur->strval = strval;
+ break;
+ }
+ default:
+ __builtin_unreachable ();
+ }
+}
+
+/* Disable a tunable if it is set. */
+static void
+disable_tunable (tunable_id_t id, char **envp)
+{
+ const char *env_alias = tunable_list[id].env_alias;
+
+ if (env_alias)
+ tunables_unsetenv (envp, tunable_list[id].env_alias);
+}
+
+/* Disable the glibc.malloc.check tunable in SETUID/SETGID programs unless
+ the system administrator overrides it by creating the /etc/suid-debug
+ file. This is a special case where we want to conditionally enable/disable
+ a tunable even for setuid binaries. We use the special version of access()
+ to avoid setting ERRNO, which is a TLS variable since TLS has not yet been
+ set up. */
+static inline void
+__always_inline
+maybe_disable_malloc_check (void)
+{
+ if (__libc_enable_secure && __access_noerrno ("/etc/suid-debug", F_OK) != 0)
+ disable_tunable (TUNABLE_ENUM_NAME(glibc, malloc, check), __environ);
+}
+
+/* Initialize the tunables list from the environment. For now we only use the
+ ENV_ALIAS to find values. Later we will also use the tunable names to find
+ values. */
+void
+__tunables_init (char **envp)
+{
+ char *envname = NULL;
+ char *envval = NULL;
+ size_t len = 0;
+
+ maybe_disable_malloc_check ();
+
+ while ((envp = get_next_env (envp, &envname, &len, &envval)) != NULL)
+ {
+ for (int i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++)
+ {
+ tunable_t *cur = &tunable_list[i];
+
+ /* Skip over tunables that have either been set already or should be
+ skipped. */
+ if (cur->strval != NULL || cur->env_alias == NULL
+ || (__libc_enable_secure && !cur->is_secure))
+ continue;
+
+ const char *name = cur->env_alias;
+
+ /* We have a match. Initialize and move on to the next line. */
+ if (is_name (name, envname))
+ {
+ tunable_initialize (cur, envval);
+ break;
+ }
+ }
+ }
+}
+
+/* Set the tunable value. This is called by the module that the tunable exists
+ in. */
+void
+__tunable_set_val (tunable_id_t id, void *valp, tunable_callback_t callback)
+{
+ tunable_t *cur = &tunable_list[id];
+
+ /* Don't do anything if our tunable was not set during initialization or if
+ it failed validation. */
+ if (cur->strval == NULL)
+ return;
+
+ if (valp == NULL)
+ goto cb;
+
+ switch (cur->type.type_code)
+ {
+ case TUNABLE_TYPE_INT_32:
+ {
+ *((int32_t *) valp) = (int32_t) cur->val.numval;
+ break;
+ }
+ case TUNABLE_TYPE_SIZE_T:
+ {
+ *((size_t *) valp) = (size_t) cur->val.numval;
+ break;
+ }
+ case TUNABLE_TYPE_STRING:
+ {
+ *((const char **)valp) = cur->val.strval;
+ break;
+ }
+ default:
+ __builtin_unreachable ();
+ }
+
+cb:
+ if (callback)
+ callback (&cur->val);
+}
diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
new file mode 100644
index 0000000..10a644b
--- /dev/null
+++ b/elf/dl-tunables.h
@@ -0,0 +1,78 @@
+/* The tunable framework. See the README to know how to use the tunable in
+ a glibc module.
+
+ Copyright (C) 2016 Free Software Foundation, Inc.
+ This file is part of the GNU C Library.
+
+ The GNU C Library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 2.1 of the License, or (at your option) any later version.
+
+ The GNU C Library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with the GNU C Library; if not, see
+ <http://www.gnu.org/licenses/>. */
+
+#ifndef _TUNABLES_H_
+# define _TUNABLES_H_
+# include <stddef.h>
+# include "dl-tunable-types.h"
+
+/* A tunable. */
+struct _tunable
+{
+ const char *name; /* Internal name of the tunable. */
+ tunable_type_t type; /* Data type of the tunable. */
+ tunable_val_t val; /* The value. */
+ const char *strval; /* The string containing the value,
+ points into envp. */
+ bool is_secure; /* Whether the tunable must be read
+ even for setuid binaries. Note that
+ even if the tunable is read, it may
+ not get used by the target module if
+ the value is considered unsafe. */
+ /* Compatibility elements. */
+ const char *env_alias; /* The compatibility environment
+ variable name. */
+};
+
+typedef struct _tunable tunable_t;
+
+/* Full name for a tunable is top_ns.tunable_ns.id. */
+#define TUNABLE_NAME_S(top,ns,id) #top "." #ns "." #id
+
+# define TUNABLE_ENUM_NAME(top,ns,id) TUNABLE_ENUM_NAME1 (top,ns,id)
+# define TUNABLE_ENUM_NAME1(top,ns,id) top ## _ ## ns ## _ ## id
+
+#include "dl-tunable-list.h"
+
+extern void __tunables_init (char **);
+extern void __tunable_set_val (tunable_id_t, void *, tunable_callback_t);
+
+/* Check if the tunable has been set to a non-default value and if it is, copy
+ it over into __VAL. */
+# define TUNABLE_SET_VAL(__id,__val) \
+({ \
+ __tunable_set_val \
+ (TUNABLE_ENUM_NAME (TOP_NAMESPACE, TUNABLE_NAMESPACE, __id), (__val), \
+ NULL); \
+})
+
+/* Same as TUNABLE_SET_VAL, but also set the callback function to __CB and call
+ it. */
+# define TUNABLE_SET_VAL_WITH_CALLBACK(__id,__val,__cb) \
+({ \
+ __tunable_set_val \
+ (TUNABLE_ENUM_NAME (TOP_NAMESPACE, TUNABLE_NAMESPACE, __id), (__val), \
+ DL_TUNABLE_CALLBACK (__cb)); \
+})
+
+/* Namespace sanity for callback functions. Use this macro to keep the
+ namespace of the modules clean. */
+#define DL_TUNABLE_CALLBACK(__name) _dl_tunable_ ## __name
+#endif
diff --git a/elf/dl-tunables.list b/elf/dl-tunables.list
new file mode 100644
index 0000000..11504c4
--- /dev/null
+++ b/elf/dl-tunables.list
@@ -0,0 +1,69 @@
+# Copyright (C) 2016 Free Software Foundation, Inc.
+# This file is part of the GNU C Library.
+
+# The GNU C Library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+
+# The GNU C Library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+
+# You should have received a copy of the GNU Lesser General Public
+# License along with the GNU C Library; if not, see
+# <http://www.gnu.org/licenses/>.
+
+# Allowed attributes for tunables:
+#
+# type: Defaults to STRING
+# minval: Optional minimum acceptable value
+# maxval: Optional maximum acceptable value
+# env_alias: An alias environment variable
+# is_secure: Specify whether the environment variable should be read for
+# setuid binaries.
+
+glibc {
+ malloc {
+ check {
+ type: INT_32
+ minval: 0
+ maxval: 3
+ env_alias: MALLOC_CHECK_
+ is_secure: true
+ }
+ top_pad {
+ type: SIZE_T
+ env_alias: MALLOC_TOP_PAD_
+ }
+ perturb {
+ type: INT_32
+ minval: 0
+ maxval: 0xff
+ env_alias: MALLOC_PERTURB_
+ }
+ mmap_threshold {
+ type: SIZE_T
+ env_alias: MALLOC_MMAP_THRESHOLD_
+ }
+ trim_threshold {
+ type: SIZE_T
+ env_alias: MALLOC_TRIM_THRESHOLD_
+ }
+ mmap_max {
+ type: INT_32
+ env_alias: MALLOC_MMAP_MAX_
+ }
+ arena_max {
+ type: SIZE_T
+ env_alias: MALLOC_ARENA_MAX
+ minval: 1
+ }
+ arena_test {
+ type: SIZE_T
+ env_alias: MALLOC_ARENA_TEST
+ minval: 1
+ }
+ }
+}
diff --git a/elf/rtld.c b/elf/rtld.c
index 647661c..eb211a2 100644
--- a/elf/rtld.c
+++ b/elf/rtld.c
@@ -2527,11 +2527,13 @@ process_envvars (enum mode *modep)
}
while (*nextp != '\0');
+#if !HAVE_TUNABLES
if (__access ("/etc/suid-debug", F_OK) != 0)
{
unsetenv ("MALLOC_CHECK_");
GLRO(dl_debug_mask) = 0;
}
+#endif
if (mode != normal)
_exit (5);
diff --git a/malloc/Makefile b/malloc/Makefile
index b8efcd6..4e4104e 100644
--- a/malloc/Makefile
+++ b/malloc/Makefile
@@ -37,6 +37,7 @@ tests := mallocbug tst-malloc tst-valloc tst-calloc tst-obstack \
tests-static := \
tst-interpose-static-nothread \
tst-interpose-static-thread \
+ tst-malloc-usable-static
tests += $(tests-static)
test-srcs = tst-mtrace
@@ -158,6 +159,7 @@ endif
tst-mcheck-ENV = MALLOC_CHECK_=3
tst-malloc-usable-ENV = MALLOC_CHECK_=3
+tst-malloc-usable-static-ENV = $(tst-malloc-usable-ENV)
# Uncomment this for test releases. For public releases it is too expensive.
#CPPFLAGS-malloc.o += -DMALLOC_DEBUG=1
diff --git a/malloc/arena.c b/malloc/arena.c
index eed4247..c2791b2 100644
--- a/malloc/arena.c
+++ b/malloc/arena.c
@@ -19,6 +19,11 @@
#include <stdbool.h>
+#if HAVE_TUNABLES
+# define TUNABLE_NAMESPACE malloc
+# include <elf/dl-tunables.h>
+#endif
+
/* Compile-time constants. */
#define HEAP_MIN_SIZE (32 * 1024)
@@ -204,6 +209,34 @@ __malloc_fork_unlock_child (void)
__libc_lock_init (list_lock);
}
+#if HAVE_TUNABLES
+static inline int do_set_mallopt_check (int32_t value);
+void
+DL_TUNABLE_CALLBACK (set_mallopt_check) (void *valp)
+{
+ int32_t value = *(int32_t *) valp;
+ do_set_mallopt_check (value);
+ if (check_action != 0)
+ __malloc_check_init ();
+}
+
+# define DL_TUNABLE_CALLBACK_FNDECL(__name, __type) \
+static inline int do_ ## __name (__type value); \
+void \
+DL_TUNABLE_CALLBACK (__name) (void *valp) \
+{ \
+ __type value = *(__type *) valp; \
+ do_ ## __name (value); \
+}
+
+DL_TUNABLE_CALLBACK_FNDECL (set_mmap_threshold, size_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_mmaps_max, int32_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_top_pad, size_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_perturb_byte, int32_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_trim_threshold, size_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_arena_max, size_t)
+DL_TUNABLE_CALLBACK_FNDECL (set_arena_test, size_t)
+#else
/* Initialization routine. */
#include <string.h>
extern char **_environ;
@@ -238,6 +271,7 @@ next_env_entry (char ***position)
return result;
}
+#endif
#ifdef SHARED
@@ -272,6 +306,24 @@ ptmalloc_init (void)
#endif
thread_arena = &main_arena;
+
+#if HAVE_TUNABLES
+ /* Ensure initialization/consolidation and do it under a lock so that a
+ thread attempting to use the arena in parallel waits on us till we
+ finish. */
+ __libc_lock_lock (main_arena.mutex);
+ malloc_consolidate (&main_arena);
+
+ TUNABLE_SET_VAL_WITH_CALLBACK (check, NULL, set_mallopt_check);
+ TUNABLE_SET_VAL_WITH_CALLBACK (top_pad, NULL, set_top_pad);
+ TUNABLE_SET_VAL_WITH_CALLBACK (perturb, NULL, set_perturb_byte);
+ TUNABLE_SET_VAL_WITH_CALLBACK (mmap_threshold, NULL, set_mmap_threshold);
+ TUNABLE_SET_VAL_WITH_CALLBACK (trim_threshold, NULL, set_trim_threshold);
+ TUNABLE_SET_VAL_WITH_CALLBACK (mmap_max, NULL, set_mmaps_max);
+ TUNABLE_SET_VAL_WITH_CALLBACK (arena_max, NULL, set_arena_max);
+ TUNABLE_SET_VAL_WITH_CALLBACK (arena_test, NULL, set_arena_test);
+ __libc_lock_unlock (main_arena.mutex);
+#else
const char *s = NULL;
if (__glibc_likely (_environ != NULL))
{
@@ -340,6 +392,8 @@ ptmalloc_init (void)
if (check_action != 0)
__malloc_check_init ();
}
+#endif
+
#if HAVE_MALLOC_INIT_HOOK
void (*hook) (void) = atomic_forced_read (__malloc_initialize_hook);
if (hook != NULL)
diff --git a/malloc/tst-malloc-usable-static.c b/malloc/tst-malloc-usable-static.c
new file mode 100644
index 0000000..8907db0
--- /dev/null
+++ b/malloc/tst-malloc-usable-static.c
@@ -0,0 +1 @@
+#include <malloc/tst-malloc-usable.c>
diff --git a/manual/install.texi b/manual/install.texi
index de1c203..ffc166d 100644
--- a/manual/install.texi
+++ b/manual/install.texi
@@ -189,6 +189,11 @@ configure with @option{--disable-werror}.
By default for x86_64, @theglibc{} is built with the vector math library.
Use this option to disable the vector math library.
+@item --enable-tunables
+Tunables support allows additional library parameters to be customized at
+runtime. This is an experimental feature and affects startup time and is thus
+disabled by default.
+
@item --build=@var{build-system}
@itemx --host=@var{host-system}
These options are for cross-compiling. If you specify both options and
diff --git a/scripts/gen-tunables.awk b/scripts/gen-tunables.awk
new file mode 100644
index 0000000..b65b5a4
--- /dev/null
+++ b/scripts/gen-tunables.awk
@@ -0,0 +1,157 @@
+# Generate dl-tunable-list.h from dl-tunables.list
+
+BEGIN {
+ tunable=""
+ ns=""
+ top_ns=""
+}
+
+# Skip over blank lines and comments.
+/^#/ {
+ next
+}
+
+/^[ \t]*$/ {
+ next
+}
+
+# Beginning of either a top namespace, tunable namespace or a tunable, decided
+# on the current value of TUNABLE, NS or TOP_NS.
+$2 == "{" {
+ if (top_ns == "") {
+ top_ns = $1
+ }
+ else if (ns == "") {
+ ns = $1
+ }
+ else if (tunable == "") {
+ tunable = $1
+ }
+ else {
+ printf ("Unexpected occurrence of '{': %s:%d\n", FILENAME, FNR)
+ exit 1
+ }
+
+ next
+}
+
+# End of either a top namespace, tunable namespace or a tunable.
+$1 == "}" {
+ if (tunable != "") {
+ # Tunables definition ended, now fill in default attributes.
+ if (!types[top_ns][ns][tunable]) {
+ types[top_ns][ns][tunable] = "STRING"
+ }
+ if (!minvals[top_ns][ns][tunable]) {
+ minvals[top_ns][ns][tunable] = "0"
+ }
+ if (!maxvals[top_ns][ns][tunable]) {
+ maxvals[top_ns][ns][tunable] = "0"
+ }
+ if (!env_alias[top_ns][ns][tunable]) {
+ env_alias[top_ns][ns][tunable] = "NULL"
+ }
+ if (!is_secure[top_ns][ns][tunable]) {
+ is_secure[top_ns][ns][tunable] = "false"
+ }
+
+ tunable = ""
+ }
+ else if (ns != "") {
+ ns = ""
+ }
+ else if (top_ns != "") {
+ top_ns = ""
+ }
+ else {
+ printf ("syntax error: extra }: %s:%d\n", FILENAME, FNR)
+ exit 1
+ }
+ next
+}
+
+# Everything else, which could either be a tunable without any attributes or a
+# tunable attribute.
+{
+ if (ns == "") {
+ printf("Line %d: Invalid tunable outside a namespace: %s\n", NR, $0)
+ exit 1
+ }
+
+ if (tunable == "") {
+ # We encountered a tunable without any attributes, so note it with a
+ # default.
+ types[top_ns][ns][$1] = "STRING"
+ next
+ }
+
+ # Otherwise, we have encountered a tunable attribute.
+ split($0, arr, ":")
+ attr = gensub(/^[ \t]+|[ \t]+$/, "", "g", arr[1])
+ val = gensub(/^[ \t]+|[ \t]+$/, "", "g", arr[2])
+
+ if (attr == "type") {
+ types[top_ns][ns][tunable] = val
+ }
+ else if (attr == "minval") {
+ minvals[top_ns][ns][tunable] = val
+ }
+ else if (attr == "maxval") {
+ maxvals[top_ns][ns][tunable] = val
+ }
+ else if (attr == "env_alias") {
+ env_alias[top_ns][ns][tunable] = sprintf("\"%s\"", val)
+ }
+ else if (attr == "is_secure") {
+ if (val == "true" || val == "false") {
+ is_secure[top_ns][ns][tunable] = val
+ }
+ else {
+ printf("Line %d: Invalid value (%s) for is_secure: %s, ", NR, val,
+ $0)
+ print("Allowed values are 'true' or 'false'")
+ exit 1
+ }
+ }
+}
+
+END {
+ if (ns != "") {
+ print "Unterminated namespace. Is a closing brace missing?"
+ exit 1
+ }
+
+ print "/* AUTOGENERATED by gen-tunables.awk. */"
+ print "#ifndef _TUNABLES_H_"
+ print "# error \"Do not include this file directly.\""
+ print "# error \"Include tunables.h instead.\""
+ print "#endif"
+
+ # Now, the enum names
+ print "\ntypedef enum"
+ print "{"
+ for (t in types) {
+ for (n in types[t]) {
+ for (m in types[t][n]) {
+ printf (" TUNABLE_ENUM_NAME(%s, %s, %s),\n", t, n, m);
+ }
+ }
+ }
+ print "} tunable_id_t;\n"
+
+ # Finally, the tunable list.
+ print "\n#ifdef TUNABLES_INTERNAL"
+ print "static tunable_t tunable_list[] = {"
+ for (t in types) {
+ for (n in types[t]) {
+ for (m in types[t][n]) {
+ printf (" {TUNABLE_NAME_S(%s, %s, %s)", t, n, m)
+ printf (", {TUNABLE_TYPE_%s, %s, %s}, {.numval = 0}, NULL, %s, %s},\n",
+ types[t][n][m], minvals[t][n][m], maxvals[t][n][m],
+ is_secure[t][n][m], env_alias[t][n][m]);
+ }
+ }
+ }
+ print "};"
+ print "#endif"
+}
diff --git a/sysdeps/mach/hurd/dl-sysdep.c b/sysdeps/mach/hurd/dl-sysdep.c
index d730f82..40b5be5 100644
--- a/sysdeps/mach/hurd/dl-sysdep.c
+++ b/sysdeps/mach/hurd/dl-sysdep.c
@@ -44,6 +44,10 @@
#include <dl-machine.h>
#include <dl-procinfo.h>
+#if HAVE_TUNABLES
+# include <dl-tunables.h>
+#endif
+
extern void __mach_init (void);
extern int _dl_argc;
@@ -143,6 +147,10 @@ _dl_sysdep_start (void **start_argptr,
__libc_enable_secure = _dl_hurd_data->flags & EXEC_SECURE;
+#if HAVE_TUNABLES
+ __tunables_init (_environ);
+#endif
+
if (_dl_hurd_data->flags & EXEC_STACK_ARGS &&
_dl_hurd_data->user_entry == 0)
_dl_hurd_data->user_entry = (vm_address_t) ENTRY_POINT;
--
2.7.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 4/4] User manual documentation for tunables Siddhesh Poyarekar
@ 2016-11-16 8:35 ` Siddhesh Poyarekar
2016-12-27 11:41 ` Florian Weimer
2016-11-16 8:35 ` [PATCH 1/4] Add framework for tunables Siddhesh Poyarekar
` (8 subsequent siblings)
10 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-16 8:35 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Read tunables values from the users using the GLIBC_TUNABLES
environment variable. The value of this variable is a colon-separated
list of name=value pairs. So a typical string would look like this:
GLIBC_TUNABLES=glibc.malloc.mmap_threshold=2048:glibc.malloc.trim_threshold=1024
* elf/dl-tunables.c: Include mman.h.
(GLIBC_TUNABLES): New macro.
(tunables_strdup): New function.
(parse_tunables): New function.
(min_strlen): New function.
(disable_tunable): Use the new functions and macro.
(__tunables_init): Likewise.
(disable_tunable): Disable tunable from GLIBC_TUNABLES.
* malloc/tst-malloc-usable-tunables.c: New test case.
* malloc/tst-malloc-usable-static-tunables.c: New test case.
* malloc/Makefile (tests, tests-static): Add tests.
---
elf/dl-tunables.c | 129 ++++++++++++++++++++++++++++-
malloc/Makefile | 6 +-
malloc/tst-malloc-usable-static-tunables.c | 1 +
malloc/tst-malloc-usable-tunables.c | 1 +
4 files changed, 135 insertions(+), 2 deletions(-)
create mode 100644 malloc/tst-malloc-usable-static-tunables.c
create mode 100644 malloc/tst-malloc-usable-tunables.c
diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
index 91340b6..79ec8ef 100644
--- a/elf/dl-tunables.c
+++ b/elf/dl-tunables.c
@@ -22,6 +22,7 @@
#include <stdbool.h>
#include <unistd.h>
#include <stdlib.h>
+#include <sys/mman.h>
#include <libc-internal.h>
#include <sysdep.h>
#include <fcntl.h>
@@ -29,7 +30,10 @@
#define TUNABLES_INTERNAL 1
#include "dl-tunables.h"
-/* Compare environment names, bounded by the name hardcoded in glibc. */
+#define GLIBC_TUNABLES "GLIBC_TUNABLES"
+
+/* Compare environment or tunable names, bounded by the name hardcoded in
+ glibc. */
static bool
is_name (const char *orig, const char *envname)
{
@@ -44,6 +48,28 @@ is_name (const char *orig, const char *envname)
return false;
}
+static char *
+tunables_strdup (const char *in)
+{
+ size_t i = 0;
+
+ while (in[i++]);
+
+ char *out = __mmap (NULL, ALIGN_UP (i, __getpagesize ()),
+ PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1,
+ 0);
+
+ if (out == MAP_FAILED)
+ return NULL;
+
+ i--;
+
+ while (i-- > 0)
+ out[i] = in[i];
+
+ return out;
+}
+
static char **
get_next_env (char **envp, char **name, size_t *namelen, char **val)
{
@@ -208,6 +234,82 @@ tunable_initialize (tunable_t *cur, const char *strval)
}
}
+static void
+parse_tunables (char *tunestr)
+{
+ if (tunestr == NULL || *tunestr == '\0')
+ return;
+
+ char *p = tunestr;
+
+ while (true)
+ {
+ char *name = p;
+ size_t len = 0;
+
+ /* First, find where the name ends. */
+ while (p[len] != '=' && p[len] != ':' && p[len] != '\0')
+ len++;
+
+ /* If we reach the end of the string before getting a valid name-value
+ pair, bail out. */
+ if (p[len] == '\0')
+ return;
+
+ /* We did not find a valid name-value pair before encountering the
+ colon. */
+ if (p[len]== ':')
+ {
+ p += len + 1;
+ continue;
+ }
+
+ p += len + 1;
+
+ char *value = p;
+ len = 0;
+
+ while (p[len] != ':' && p[len] != '\0')
+ len++;
+
+ char end = p[len];
+ p[len] = '\0';
+
+ /* Add the tunable if it exists. */
+ for (size_t i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++)
+ {
+ tunable_t *cur = &tunable_list[i];
+
+ /* If we are in a secure context (AT_SECURE) then ignore the tunable
+ unless it is explicitly marked as secure. Tunable values take
+ precendence over their envvar aliases. */
+ if (__libc_enable_secure && !cur->is_secure)
+ continue;
+
+ if (is_name (cur->name, name))
+ {
+ tunable_initialize (cur, value);
+ break;
+ }
+ }
+
+ if (end == ':')
+ p += len + 1;
+ else
+ return;
+ }
+}
+
+static size_t
+min_strlen (const char *s)
+{
+ size_t i = 0;
+ while (*s++ != '\0')
+ i++;
+
+ return i;
+}
+
/* Disable a tunable if it is set. */
static void
disable_tunable (tunable_id_t id, char **envp)
@@ -216,6 +318,23 @@ disable_tunable (tunable_id_t id, char **envp)
if (env_alias)
tunables_unsetenv (envp, tunable_list[id].env_alias);
+
+ char *tunable = getenv (GLIBC_TUNABLES);
+ const char *cmp = tunable_list[id].name;
+ const size_t len = min_strlen (cmp);
+
+ while (tunable && *tunable != '\0' && *tunable != ':')
+ {
+ if (is_name (tunable, cmp))
+ {
+ tunable += len;
+ /* Overwrite the = and the value with colons. */
+ while (*tunable != '\0' && *tunable != ':')
+ *tunable++ = ':';
+ break;
+ }
+ tunable++;
+ }
}
/* Disable the glibc.malloc.check tunable in SETUID/SETGID programs unless
@@ -246,6 +365,14 @@ __tunables_init (char **envp)
while ((envp = get_next_env (envp, &envname, &len, &envval)) != NULL)
{
+ if (is_name (GLIBC_TUNABLES, envname))
+ {
+ char *val = tunables_strdup (envval);
+ if (val != NULL)
+ parse_tunables (val);
+ continue;
+ }
+
for (int i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++)
{
tunable_t *cur = &tunable_list[i];
diff --git a/malloc/Makefile b/malloc/Makefile
index 4e4104e..a34e20c 100644
--- a/malloc/Makefile
+++ b/malloc/Makefile
@@ -33,11 +33,13 @@ tests := mallocbug tst-malloc tst-valloc tst-calloc tst-obstack \
tst-mallocfork2 \
tst-interpose-nothread \
tst-interpose-thread \
+ tst-malloc-usable-tunables
tests-static := \
tst-interpose-static-nothread \
tst-interpose-static-thread \
- tst-malloc-usable-static
+ tst-malloc-usable-static \
+ tst-malloc-usable-static-tunables
tests += $(tests-static)
test-srcs = tst-mtrace
@@ -160,6 +162,8 @@ endif
tst-mcheck-ENV = MALLOC_CHECK_=3
tst-malloc-usable-ENV = MALLOC_CHECK_=3
tst-malloc-usable-static-ENV = $(tst-malloc-usable-ENV)
+tst-malloc-usable-tunables-ENV = GLIBC_TUNABLES=glibc.malloc.check=3
+tst-malloc-usable-static-tunables-ENV = $(tst-malloc-usable-tunables-ENV)
# Uncomment this for test releases. For public releases it is too expensive.
#CPPFLAGS-malloc.o += -DMALLOC_DEBUG=1
diff --git a/malloc/tst-malloc-usable-static-tunables.c b/malloc/tst-malloc-usable-static-tunables.c
new file mode 100644
index 0000000..8907db0
--- /dev/null
+++ b/malloc/tst-malloc-usable-static-tunables.c
@@ -0,0 +1 @@
+#include <malloc/tst-malloc-usable.c>
diff --git a/malloc/tst-malloc-usable-tunables.c b/malloc/tst-malloc-usable-tunables.c
new file mode 100644
index 0000000..8907db0
--- /dev/null
+++ b/malloc/tst-malloc-usable-tunables.c
@@ -0,0 +1 @@
+#include <malloc/tst-malloc-usable.c>
--
2.7.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (2 preceding siblings ...)
2016-11-16 8:35 ` [PATCH 1/4] Add framework for tunables Siddhesh Poyarekar
@ 2016-11-16 8:35 ` Siddhesh Poyarekar
2016-12-27 11:44 ` Florian Weimer
2016-11-21 2:58 ` [ping][PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (6 subsequent siblings)
10 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-16 8:35 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
At the GNU Tools Cauldron 2016, the state of the current tunables
patchset was considered OK with the addition of a way to select the
frontend to be used for the tunables. That is, to avoid being locked
in to one type of frontend initially, it should be possible to build
tunables with a different frontend with something as simple as a
configure switch.
To that effect, this patch enhances the --enable-tunables option to
accept more values than just 'yes' or 'no'. The current frontend (and
default when enable-tunables is 'yes') is called 'valstring', to
select the frontend where a single environment variable is set to a
colon-separated value string. More such frontends can be added in
future.
* Makeconfig (have-tunables): Check for non-negative instead
of positive.
* configure.ac: Add 'valstring' as a valid value for
--enable-tunables.
* configure: Regenerate.
* elf/Makefile (have-tunables): Check for non-negative instead
of positive.
(CPPFLAGS-dl-tunables.c): Define TUNABLES_FRONTEND for
dl-tunables.c.
* elf/dl-tunables.c (GLIBC_TUNABLES): Define only when
TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring.
(tunables_strdup): Likewise.
(disable_tunables): Likewise.
(parse_tunables): Likewise.
(__tunables_init): Process GLIBC_TUNABLES envvar only when.
TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring.
* elf/dl-tunables.h (TUNABLES_FRONTEND_valstring): New macro.
(TUNABLES_FRONTEND_yes): New macro, define as
TUNABLES_FRONTEND_valstring by default.
* manual/install.texi: Document new acceptable values for
--enable-tunables.
* INSTALL: Regenerate.
---
INSTALL | 15 ++++++++++++++-
Makeconfig | 4 ++--
configure | 3 ++-
configure.ac | 2 +-
elf/Makefile | 4 +++-
elf/dl-tunables.c | 12 +++++++++++-
elf/dl-tunables.h | 4 ++++
manual/install.texi | 18 +++++++++++++++++-
8 files changed, 54 insertions(+), 8 deletions(-)
diff --git a/INSTALL b/INSTALL
index ccacccc..55f6a23 100644
--- a/INSTALL
+++ b/INSTALL
@@ -161,7 +161,20 @@ will be used, and CFLAGS sets optimization options for the compiler.
'--enable-tunables'
Tunables support allows additional library parameters to be
customized at runtime. This is an experimental feature and affects
- startup time and is thus disabled by default.
+ startup time and is thus disabled by default. This option can take
+ the following values:
+
+ * NO This is the default if the option is not passed to
+ configure. This disables tunables.
+
+ * YES This is the default if the option is passed to configure.
+ This enables tunables and selects the default frontend
+ (currently VALSTRING).
+
+ * VALSTRING This enables tunables and selects the VALSTRING
+ frontend for tunables. This frontend allows users to specify
+ tunables as a colon-separated list in a single environment
+ variable GLIBC_TUNABLES.
'--build=BUILD-SYSTEM'
'--host=HOST-SYSTEM'
diff --git a/Makeconfig b/Makeconfig
index d0eb832..09ddb12 100644
--- a/Makeconfig
+++ b/Makeconfig
@@ -884,7 +884,7 @@ CPPFLAGS = $(config-extra-cppflags) $(CPPUNDEFS) $(CPPFLAGS-config) \
$(libof-$(<F)) $(libof-$(@F)),$(CPPFLAGS-$(lib))) \
$(CPPFLAGS-$(<F)) $(CPPFLAGS-$(@F)) $(CPPFLAGS-$(basename $(@F)))
-ifeq (yes,$(have-tunables))
+ifneq (no,$(have-tunables))
CPPFLAGS += -DTOP_NAMESPACE=glibc
endif
@@ -1064,7 +1064,7 @@ endif
# Build the tunables list header early since it could be used by any module in
# glibc.
-ifeq (yes,$(have-tunables))
+ifneq (no,$(have-tunables))
before-compile += $(common-objpfx)dl-tunable-list.h
$(common-objpfx)dl-tunable-list.h: $(..)scripts/gen-tunables.awk \
diff --git a/configure b/configure
index cbd5fb3..35a9ecf 100755
--- a/configure
+++ b/configure
@@ -1445,7 +1445,8 @@ Optional Features:
--disable-build-nscd disable building and installing the nscd daemon
--disable-nscd library functions will not contact the nscd daemon
--enable-pt_chown Enable building and installing pt_chown
- --enable-tunables Enable tunables support
+ --enable-tunables Enable tunables support. Known values are 'yes',
+ 'no' and 'valstring'
--enable-mathvec Enable building and installing mathvec [default
depends on architecture]
diff --git a/configure.ac b/configure.ac
index 3b7a0db..5e2c8ca 100644
--- a/configure.ac
+++ b/configure.ac
@@ -397,7 +397,7 @@ fi
AC_ARG_ENABLE([tunables],
[AS_HELP_STRING([--enable-tunables],
- [Enable tunables support])],
+ [Enable tunables support. Known values are 'yes', 'no' and 'valstring'])],
[have_tunables=$enableval],
[have_tunables=no])
AC_SUBST(have_tunables)
diff --git a/elf/Makefile b/elf/Makefile
index 477f8fc..34fc657 100644
--- a/elf/Makefile
+++ b/elf/Makefile
@@ -36,8 +36,10 @@ ifeq (yes,$(use-ldconfig))
dl-routines += dl-cache
endif
-ifeq (yes,$(have-tunables))
+ifneq (no,$(have-tunables))
dl-routines += dl-tunables
+tunables-type = $(addprefix TUNABLES_FRONTEND_,$(have-tunables))
+CPPFLAGS-dl-tunables.c = -DTUNABLES_FRONTEND=$(tunables-type)
endif
all-dl-routines = $(dl-routines) $(sysdep-dl-routines)
diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
index 79ec8ef..128da29 100644
--- a/elf/dl-tunables.c
+++ b/elf/dl-tunables.c
@@ -30,7 +30,9 @@
#define TUNABLES_INTERNAL 1
#include "dl-tunables.h"
-#define GLIBC_TUNABLES "GLIBC_TUNABLES"
+#if TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring
+# define GLIBC_TUNABLES "GLIBC_TUNABLES"
+#endif
/* Compare environment or tunable names, bounded by the name hardcoded in
glibc. */
@@ -48,6 +50,7 @@ is_name (const char *orig, const char *envname)
return false;
}
+#if TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring
static char *
tunables_strdup (const char *in)
{
@@ -69,6 +72,7 @@ tunables_strdup (const char *in)
return out;
}
+#endif
static char **
get_next_env (char **envp, char **name, size_t *namelen, char **val)
@@ -234,6 +238,7 @@ tunable_initialize (tunable_t *cur, const char *strval)
}
}
+#if TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring
static void
parse_tunables (char *tunestr)
{
@@ -299,6 +304,7 @@ parse_tunables (char *tunestr)
return;
}
}
+#endif
static size_t
min_strlen (const char *s)
@@ -319,6 +325,7 @@ disable_tunable (tunable_id_t id, char **envp)
if (env_alias)
tunables_unsetenv (envp, tunable_list[id].env_alias);
+#if TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring
char *tunable = getenv (GLIBC_TUNABLES);
const char *cmp = tunable_list[id].name;
const size_t len = min_strlen (cmp);
@@ -335,6 +342,7 @@ disable_tunable (tunable_id_t id, char **envp)
}
tunable++;
}
+#endif
}
/* Disable the glibc.malloc.check tunable in SETUID/SETGID programs unless
@@ -365,6 +373,7 @@ __tunables_init (char **envp)
while ((envp = get_next_env (envp, &envname, &len, &envval)) != NULL)
{
+#if TUNABLES_FRONTEND == TUNABLES_FRONTEND_valstring
if (is_name (GLIBC_TUNABLES, envname))
{
char *val = tunables_strdup (envval);
@@ -372,6 +381,7 @@ __tunables_init (char **envp)
parse_tunables (val);
continue;
}
+#endif
for (int i = 0; i < sizeof (tunable_list) / sizeof (tunable_t); i++)
{
diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
index 10a644b..27b1b64 100644
--- a/elf/dl-tunables.h
+++ b/elf/dl-tunables.h
@@ -75,4 +75,8 @@ extern void __tunable_set_val (tunable_id_t, void *, tunable_callback_t);
/* Namespace sanity for callback functions. Use this macro to keep the
namespace of the modules clean. */
#define DL_TUNABLE_CALLBACK(__name) _dl_tunable_ ## __name
+
+# define TUNABLES_FRONTEND_valstring 1
+/* The default value for TUNABLES_FRONTEND. */
+# define TUNABLES_FRONTEND_yes TUNABLES_FRONTEND_valstring
#endif
diff --git a/manual/install.texi b/manual/install.texi
index ffc166d..e34c7bf 100644
--- a/manual/install.texi
+++ b/manual/install.texi
@@ -192,7 +192,23 @@ Use this option to disable the vector math library.
@item --enable-tunables
Tunables support allows additional library parameters to be customized at
runtime. This is an experimental feature and affects startup time and is thus
-disabled by default.
+disabled by default. This option can take the following values:
+
+@itemize @bullet
+@item @var{no}
+This is the default if the option is not passed to configure. This disables
+tunables.
+
+@item @var{yes}
+This is the default if the option is passed to configure. This enables tunables
+and selects the default frontend (currently @var{valstring}).
+
+@item @var{valstring}
+This enables tunables and selects the @var{valstring} frontend for tunables.
+This frontend allows users to specify tunables as a colon-separated list in a
+single environment variable @var{GLIBC_TUNABLES}.
+
+@end itemize
@item --build=@var{build-system}
@itemx --host=@var{host-system}
--
2.7.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PATCH 4/4] User manual documentation for tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
@ 2016-11-16 8:35 ` Siddhesh Poyarekar
2016-11-16 13:49 ` Rical Jasan
2016-11-16 8:35 ` [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable Siddhesh Poyarekar
` (9 subsequent siblings)
10 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-16 8:35 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Create a new node for tunables documentation and add notes for the
malloc tunables.
* manual/tunables.texi: New chapter.
* manual/Makefile (chapters): Add it.
* manual/probes.texi (@node): Point to the Tunables chapter.
---
manual/Makefile | 3 +-
manual/probes.texi | 2 +-
manual/tunables.texi | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 188 insertions(+), 2 deletions(-)
create mode 100644 manual/tunables.texi
diff --git a/manual/Makefile b/manual/Makefile
index f2f694f..ecc2bf6 100644
--- a/manual/Makefile
+++ b/manual/Makefile
@@ -38,7 +38,8 @@ chapters = $(addsuffix .texi, \
message search pattern io stdio llio filesys \
pipe socket terminal syslog math arith time \
resource setjmp signal startup process ipc job \
- nss users sysinfo conf crypt debug threads probes)
+ nss users sysinfo conf crypt debug threads \
+ probes tunables)
add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi))
appendices = lang.texi header.texi install.texi maint.texi platform.texi \
contrib.texi
diff --git a/manual/probes.texi b/manual/probes.texi
index 237a918..eb91c62 100644
--- a/manual/probes.texi
+++ b/manual/probes.texi
@@ -1,5 +1,5 @@
@node Internal Probes
-@c @node Internal Probes, , POSIX Threads, Top
+@c @node Internal Probes, Tunables, POSIX Threads, Top
@c %MENU% Probes to monitor libc internal behavior
@chapter Internal probes
diff --git a/manual/tunables.texi b/manual/tunables.texi
new file mode 100644
index 0000000..1f1f9fc
--- /dev/null
+++ b/manual/tunables.texi
@@ -0,0 +1,185 @@
+@node Tunables
+@c @node Tunables, , Internal Probes, Top
+@c %MENU% Tunable switches to alter libc internal behavior
+@chapter Tunables
+
+Tunables is a feature in @theglibc{} that allows application authors and
+distribution maintainers to alter the runtime library behaviour to match
+their workload. These are implemented as a set of switches that may be
+modified in different ways. The current default method to do this is via
+the @env{GLIBC_TUNABLES} environment variable by setting it to a string
+of colon-separated @var{name}=@var{value} pairs. For example, the following
+environment variable export enables malloc checking and sets the malloc
+trim threshold to 128 bytes.
+
+@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3}
+
+Tunables are not part of the @glibcadj{} stable ABI, and they are
+subject to change or removal across releases. Additionally, the method to
+modify tunable values may change between releases and across distributions.
+It is possible to implement multiple `frontends' for the tunables allowing
+distributions to choose their preferred method at build time.
+
+Finally, the set of tunables available may vary between distributions as
+the tunables feature allows distributions to add their own tunables under
+their own namespace.
+
+@menu
+* Tunable names:: The structure of a tunable name
+* Memory Allocation Tunables:: Tunables in the memory allocation subsystem
+@end menu
+
+@node Tunable names
+@section Tunable names
+
+A tunable name is split into three components, a top namespace, a tunable
+namespace and the tunable name. The top namespace for tunables
+implemented in @theglibc{} project is @code{glibc}. Distributions that
+choose to add custom tunables in their maintained versions of @theglibc{}
+may choose to do so under their own top namespace.
+
+The tunable namespace is a logical grouping of tunables in a single
+module. This currently holds no special significance, although that may
+change in future.
+
+The tunable name is the actual name of the tunable. It is possible that
+different tunable namespaces may have tunables within them that have the
+same name, likewise for top namespaces. Hence, we only support
+identification of tunables by their full name, i.e. with the top
+namespace, tunable namespace and tunable name, separated by periods.
+
+@node Memory Allocation Tunables
+@section Memory Allocation Tunables
+
+@deftp Namespace glibc.malloc
+Memory allocation behaviour can be modified by setting any of the
+following tunables in the @code{malloc} namespace.
+@end deftp
+
+@deftp Tunable glibc.malloc.check
+This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is
+identical in features.
+
+Setting this tunable enables a special (less efficient) memory allocator for
+the malloc family of functions that is designed to be tolerant against simple
+errors such as double calls of free with the same argument, or overruns of a
+single byte (off-by-one bugs). Not all such errors can be protected against,
+however, and memory leaks can result. The following list describes the values
+that this tunable can take and the effect they have on malloc functionality.
+
+@itemize @bullet
+@item @code{0} Disable all error reporting. The alternate allocator is
+selected and heap corruption detection is in place, but any such errors
+detected are ignored. This is currently a supported use, but is not
+recommended.
+@item @code{1} Report errors. The alternate allocator is selected and heap
+corruption, if detected, is reported as diagnostic messages to @code{stderr}
+and the program continues execution.
+@item @code{2} Abort on errors. The alternate allocator is selected and if
+heap corruption is detected, the program is ended immediately by calling
+@code{abort}.
+@item @code{3} Fully enabled. The alternate allocator is selected and is fully
+functional. That is, if heap corruption is detected, a verbose diagnostic
+message is printed to @code{stderr} and the program is ended by calling
+@code{abort}.
+@end itemize
+
+Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it
+diverges from normal program behavior by writing to @code{stderr}, which could
+by exploited in SUID and SGID binaries. Therefore, @code{glibc.malloc.check}
+is disabled by default for SUID and SGID binaries. This can be enabled again
+by the system administrator by adding a file @file{/etc/suid-debug}; the
+content of the file could be anything or even empty.
+@end deftp
+
+@deftp Tunable glibc.malloc.top_pad
+This tunable supersedes the @code{MALLOC_TOP_PAD_} environment variable and is
+identical in features.
+
+This tunable determines the amount of extra memory to obtain from the system
+when any of the arenas need to be extended. It also specifies the number of
+bytes to retain when shrinking any of the arenas. This provides the necessary
+hysteresis in heap size such that excessive amounts of system calls can be
+avoided.
+
+The default value of this tunable is @samp{0}.
+@end deftp
+
+@deftp Tunable glibc.malloc.perturb
+This tunable supersedes the @code{MALLOC_PERTURB_} environment variable and is
+identical in features.
+
+If set to a non-zero value, memory blocks are initialized with values depending
+on some low order bits of this tunable when they are allocated (except when
+allocated by calloc) and freed. This can be used to debug the use of
+uninitialized or freed heap memory. Note that this option does not guarantee
+that the freed block will have any specific values. It only guarantees that the
+content the block had before it was freed will be overwritten.
+
+The default value of this tunable is @samp{0}.
+@end deftp
+
+@deftp Tunable glibc.malloc.mmap_threshold
+This tunable supersedes the @code{MALLOC_MMAP_THRESHOLD_} environment variable
+and is identical in features.
+
+When this tunable is set, all chunks larger than this value are allocated
+outside the normal heap, using the @code{mmap} system call. This way it is
+guaranteed that the memory for these chunks can be returned to the system on
+@code{free}. Note that requests smaller than this threshold might still be
+allocated via @code{mmap}.
+
+If this tunable is not set, the default value is set as 128 KB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program. If the tunable is set, the dynamic adjustment is disabled and the
+value is set as static.
+@end deftp
+
+@deftp Tunable glibc.malloc.trim_threshold
+This tunable supersedes the @code{MALLOC_TRIM_THRESHOLD_} environment variable
+and is identical in features.
+
+The value of this tunable is the minimum size (in bytes) of the top-most,
+releasable chunk in an arena that will trigger a system call in order to return
+memory to the system from that arena.
+
+If this tunable is not set, the default value is set as 128 KB and the
+threshold is adjusted dynamically to suit the allocation patterns of the
+program. If the tunable is set, the dynamic adjustment is disabled and the
+value is set as static.
+@end deftp
+
+@deftp Tunable glibc.malloc.mmap_max
+This tunable supersedes the @code{MALLOC_MMAP_MAX_} environment variable and is
+identical in features.
+
+The value of this tunable is maximum number of chunks to allocate with
+@code{mmap}. Setting this to zero disables all use of @code{mmap}.
+
+The default value of this tunable is @samp{65536}.
+@end deftp
+
+@deftp Tunable glibc.malloc.arena_test
+This tunable supersedes the @code{MALLOC_ARENA_TEST} environment variable and is
+identical in features.
+
+The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that
+can be created before the test on the limit to the number of arenas is
+conducted. The value is ignored if @code{glibc.malloc.arena_max} is set.
+
+The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit
+systems.
+@end deftp
+
+@deftp Tunable glibc.malloc.arena_max
+This tunable supersedes the @code{MALLOC_ARENA_MAX} environment variable and is
+identical in features.
+
+This tunable sets the number of arenas to use in a process regardless of the
+number of cores in the system.
+
+The default value of this tunable is @code{0}, meaning that the limit on the
+number of arenas is determined by the number of CPU cores online. For 32-bit
+systems the limit is twice the number of cores online and on 64-bit systems, it
+is 8 times the number of cores online.
+@end deftp
--
2.7.4
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 4/4] User manual documentation for tunables
2016-11-16 8:35 ` [PATCH 4/4] User manual documentation for tunables Siddhesh Poyarekar
@ 2016-11-16 13:49 ` Rical Jasan
2016-11-21 3:07 ` Siddhesh Poyarekar
0 siblings, 1 reply; 26+ messages in thread
From: Rical Jasan @ 2016-11-16 13:49 UTC (permalink / raw)
To: Siddhesh Poyarekar, libc-alpha; +Cc: carlos, adhemerval.zanella
On 11/16/2016 12:35 AM, Siddhesh Poyarekar wrote:
> Create a new node for tunables documentation and add notes for the
> malloc tunables.
>
> * manual/tunables.texi: New chapter.
> * manual/Makefile (chapters): Add it.
> * manual/probes.texi (@node): Point to the Tunables chapter.
> ---
> manual/Makefile | 3 +-
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 188 insertions(+), 2 deletions(-)
> create mode 100644 manual/tunables.texi
>
> diff --git a/manual/Makefile b/manual/Makefile
> index f2f694f..ecc2bf6 100644
> --- a/manual/Makefile
> +++ b/manual/Makefile
> @@ -38,7 +38,8 @@ chapters = $(addsuffix .texi, \
> message search pattern io stdio llio filesys \
> pipe socket terminal syslog math arith time \
> resource setjmp signal startup process ipc job \
> - nss users sysinfo conf crypt debug threads probes)
> + nss users sysinfo conf crypt debug threads \
> + probes tunables)
> add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi))
> appendices = lang.texi header.texi install.texi maint.texi platform.texi \
> contrib.texi
> diff --git a/manual/probes.texi b/manual/probes.texi
> index 237a918..eb91c62 100644
> --- a/manual/probes.texi
> +++ b/manual/probes.texi
> @@ -1,5 +1,5 @@
> @node Internal Probes
> -@c @node Internal Probes, , POSIX Threads, Top
> +@c @node Internal Probes, Tunables, POSIX Threads, Top
> @c %MENU% Probes to monitor libc internal behavior
> @chapter Internal probes
>
> diff --git a/manual/tunables.texi b/manual/tunables.texi
> new file mode 100644
> index 0000000..1f1f9fc
> --- /dev/null
> +++ b/manual/tunables.texi
> @@ -0,0 +1,185 @@
> +@node Tunables
> +@c @node Tunables, , Internal Probes, Top
> +@c %MENU% Tunable switches to alter libc internal behavior
> +@chapter Tunables
Maybe some Concept Index entries here? At least:
@cindex tunables
> +
> +Tunables is a feature in @theglibc{} that allows application authors and
@dfn{Tunables} are
> +distribution maintainers to alter the runtime library behaviour to match
> +their workload. These are implemented as a set of switches that may be
> +modified in different ways. The current default method to do this is via
> +the @env{GLIBC_TUNABLES} environment variable by setting it to a string
> +of colon-separated @var{name}=@var{value} pairs. For example, the following
> +environment variable export enables malloc checking and sets the malloc
> +trim threshold to 128 bytes.
Anything with "the following", I usually end with a ":".
> +
> +@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3}
Could be a @smallexample, and omit the @code{}.
> +
> +Tunables are not part of the @glibcadj{} stable ABI, and they are
> +subject to change or removal across releases. Additionally, the method to
> +modify tunable values may change between releases and across distributions.
> +It is possible to implement multiple `frontends' for the tunables allowing
> +distributions to choose their preferred method at build time.
> +
> +Finally, the set of tunables available may vary between distributions as
> +the tunables feature allows distributions to add their own tunables under
> +their own namespace.
> +
> +@menu
> +* Tunable names:: The structure of a tunable name
> +* Memory Allocation Tunables:: Tunables in the memory allocation subsystem
> +@end menu
> +
> +@node Tunable names
> +@section Tunable names
More proposed @cindex entries:
@cindex tunable names
@cindex tunable namespaces
> +
> +A tunable name is split into three components, a top namespace, a tunable
> +namespace and the tunable name. The top namespace for tunables
> +implemented in @theglibc{} project is @code{glibc}. Distributions that
-project
> +choose to add custom tunables in their maintained versions of @theglibc{}
> +may choose to do so under their own top namespace.
> +
> +The tunable namespace is a logical grouping of tunables in a single
> +module. This currently holds no special significance, although that may
> +change in future.
the future
> +
> +The tunable name is the actual name of the tunable. It is possible that
> +different tunable namespaces may have tunables within them that have the
> +same name, likewise for top namespaces. Hence, we only support
> +identification of tunables by their full name, i.e. with the top
> +namespace, tunable namespace and tunable name, separated by periods.
> +
> +@node Memory Allocation Tunables
> +@section Memory Allocation Tunables
Some @cindex entries here would be good, particularly tying together
malloc + tunables; e.g.:
@cindex memory allocation tunables
@cindex malloc, tunables
@cindex tunables, malloc
> +
> +@deftp Namespace glibc.malloc
> +Memory allocation behaviour can be modified by setting any of the
> +following tunables in the @code{malloc} namespace.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.check
> +This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is
> +identical in features.
> +
> +Setting this tunable enables a special (less efficient) memory allocator for
> +the malloc family of functions that is designed to be tolerant against simple
> +errors such as double calls of free with the same argument, or overruns of a
> +single byte (off-by-one bugs). Not all such errors can be protected against,
> +however, and memory leaks can result. The following list describes the values
> +that this tunable can take and the effect they have on malloc functionality.
> +
> +@itemize @bullet
> +@item @code{0} Disable all error reporting. The alternate allocator is
> +selected and heap corruption detection is in place, but any such errors
> +detected are ignored. This is currently a supported use, but is not
> +recommended.
> +@item @code{1} Report errors. The alternate allocator is selected and heap
> +corruption, if detected, is reported as diagnostic messages to @code{stderr}
> +and the program continues execution.
> +@item @code{2} Abort on errors. The alternate allocator is selected and if
> +heap corruption is detected, the program is ended immediately by calling
> +@code{abort}.
> +@item @code{3} Fully enabled. The alternate allocator is selected and is fully
> +functional. That is, if heap corruption is detected, a verbose diagnostic
> +message is printed to @code{stderr} and the program is ended by calling
> +@code{abort}.
> +@end itemize
> +
> +Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it
> +diverges from normal program behavior by writing to @code{stderr}, which could
> +by exploited in SUID and SGID binaries. Therefore, @code{glibc.malloc.check}
> +is disabled by default for SUID and SGID binaries. This can be enabled again
> +by the system administrator by adding a file @file{/etc/suid-debug}; the
> +content of the file could be anything or even empty.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.top_pad
> +This tunable supersedes the @code{MALLOC_TOP_PAD_} environment variable and is
> +identical in features.
> +
> +This tunable determines the amount of extra memory to obtain from the system
"amount of extra memory in bytes..."; I think it would be nice if the
first sentence mentioned the units.
> +when any of the arenas need to be extended. It also specifies the number of
> +bytes to retain when shrinking any of the arenas. This provides the necessary
> +hysteresis in heap size such that excessive amounts of system calls can be
> +avoided.
> +
> +The default value of this tunable is @samp{0}.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.perturb
> +This tunable supersedes the @code{MALLOC_PERTURB_} environment variable and is
> +identical in features.
> +
> +If set to a non-zero value, memory blocks are initialized with values depending
> +on some low order bits of this tunable when they are allocated (except when
> +allocated by calloc) and freed. This can be used to debug the use of
> +uninitialized or freed heap memory. Note that this option does not guarantee
> +that the freed block will have any specific values. It only guarantees that the
> +content the block had before it was freed will be overwritten.
This seems non-deterministic and leaves me wondering which low-order
bits, and how to select my values so I know what gets written.
> +
> +The default value of this tunable is @samp{0}.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.mmap_threshold
> +This tunable supersedes the @code{MALLOC_MMAP_THRESHOLD_} environment variable
> +and is identical in features.
> +
> +When this tunable is set, all chunks larger than this value are allocated
> +outside the normal heap, using the @code{mmap} system call. This way it is
> +guaranteed that the memory for these chunks can be returned to the system on
> +@code{free}. Note that requests smaller than this threshold might still be
> +allocated via @code{mmap}.
Units here, too.
> +
> +If this tunable is not set, the default value is set as 128 KB and the
> +threshold is adjusted dynamically to suit the allocation patterns of the
> +program. If the tunable is set, the dynamic adjustment is disabled and the
> +value is set as static.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.trim_threshold
> +This tunable supersedes the @code{MALLOC_TRIM_THRESHOLD_} environment variable
> +and is identical in features.
> +
> +The value of this tunable is the minimum size (in bytes) of the top-most,
> +releasable chunk in an arena that will trigger a system call in order to return
> +memory to the system from that arena.
> +
> +If this tunable is not set, the default value is set as 128 KB and the
set to
I thought this was in bytes! ;)
> +threshold is adjusted dynamically to suit the allocation patterns of the
> +program. If the tunable is set, the dynamic adjustment is disabled and the
> +value is set as static.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.mmap_max
> +This tunable supersedes the @code{MALLOC_MMAP_MAX_} environment variable and is
> +identical in features.
> +
> +The value of this tunable is maximum number of chunks to allocate with
> +@code{mmap}. Setting this to zero disables all use of @code{mmap}.
> +
> +The default value of this tunable is @samp{65536}.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.arena_test
> +This tunable supersedes the @code{MALLOC_ARENA_TEST} environment variable and is
> +identical in features.
> +
> +The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that
> +can be created before the test on the limit to the number of arenas is
> +conducted. The value is ignored if @code{glibc.malloc.arena_max} is set.
> +
> +The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit
> +systems.
> +@end deftp
> +
> +@deftp Tunable glibc.malloc.arena_max
> +This tunable supersedes the @code{MALLOC_ARENA_MAX} environment variable and is
> +identical in features.
> +
> +This tunable sets the number of arenas to use in a process regardless of the
> +number of cores in the system.
> +
> +The default value of this tunable is @code{0}, meaning that the limit on the
> +number of arenas is determined by the number of CPU cores online. For 32-bit
> +systems the limit is twice the number of cores online and on 64-bit systems, it
> +is 8 times the number of cores online.
> +@end deftp
Whitespace issues throughout; noticed a lot of single-space separated
sentences. Not sure that it matters in the rendered output, but it's
nice for source readability.
I like how you made this its own chapter. I think it's a good start.
Rical
^ permalink raw reply [flat|nested] 26+ messages in thread
* [ping][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (3 preceding siblings ...)
2016-11-16 8:35 ` [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time Siddhesh Poyarekar
@ 2016-11-21 2:58 ` Siddhesh Poyarekar
2016-11-24 13:31 ` [PING 2][PATCHv6 " Siddhesh Poyarekar
` (5 subsequent siblings)
10 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-21 2:58 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 4/4] User manual documentation for tunables
2016-11-16 13:49 ` Rical Jasan
@ 2016-11-21 3:07 ` Siddhesh Poyarekar
0 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-21 3:07 UTC (permalink / raw)
To: Rical Jasan, libc-alpha; +Cc: carlos, adhemerval.zanella
On Wednesday 16 November 2016 07:19 PM, Rical Jasan wrote:
>> diff --git a/manual/tunables.texi b/manual/tunables.texi
>> new file mode 100644
>> index 0000000..1f1f9fc
>> --- /dev/null
>> +++ b/manual/tunables.texi
>> @@ -0,0 +1,185 @@
>> +@node Tunables
>> +@c @node Tunables, , Internal Probes, Top
>> +@c %MENU% Tunable switches to alter libc internal behavior
>> +@chapter Tunables
>
> Maybe some Concept Index entries here? At least:
>
> @cindex tunables
OK
>> +
>> +Tunables is a feature in @theglibc{} that allows application authors and
>
> @dfn{Tunables} are
OK
>> +distribution maintainers to alter the runtime library behaviour to match
>> +their workload. These are implemented as a set of switches that may be
>> +modified in different ways. The current default method to do this is via
>> +the @env{GLIBC_TUNABLES} environment variable by setting it to a string
>> +of colon-separated @var{name}=@var{value} pairs. For example, the following
>> +environment variable export enables malloc checking and sets the malloc
>> +trim threshold to 128 bytes.
>
> Anything with "the following", I usually end with a ":".
>
I do too usually, thanks :)
>> +
>> +@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3}
>
> Could be a @smallexample, and omit the @code{}.
OK
>
>> +
>> +Tunables are not part of the @glibcadj{} stable ABI, and they are
>> +subject to change or removal across releases. Additionally, the method to
>> +modify tunable values may change between releases and across distributions.
>> +It is possible to implement multiple `frontends' for the tunables allowing
>> +distributions to choose their preferred method at build time.
>> +
>> +Finally, the set of tunables available may vary between distributions as
>> +the tunables feature allows distributions to add their own tunables under
>> +their own namespace.
>> +
>> +@menu
>> +* Tunable names:: The structure of a tunable name
>> +* Memory Allocation Tunables:: Tunables in the memory allocation subsystem
>> +@end menu
>> +
>> +@node Tunable names
>> +@section Tunable names
>
> More proposed @cindex entries:
>
> @cindex tunable names
> @cindex tunable namespaces
>
OK
>> +
>> +A tunable name is split into three components, a top namespace, a tunable
>> +namespace and the tunable name. The top namespace for tunables
>> +implemented in @theglibc{} project is @code{glibc}. Distributions that
>
> -project
>
OK
>> +choose to add custom tunables in their maintained versions of @theglibc{}
>> +may choose to do so under their own top namespace.
>> +
>> +The tunable namespace is a logical grouping of tunables in a single
>> +module. This currently holds no special significance, although that may
>> +change in future.
>
> the future
>
OK
>> +
>> +The tunable name is the actual name of the tunable. It is possible that
>> +different tunable namespaces may have tunables within them that have the
>> +same name, likewise for top namespaces. Hence, we only support
>> +identification of tunables by their full name, i.e. with the top
>> +namespace, tunable namespace and tunable name, separated by periods.
>> +
>> +@node Memory Allocation Tunables
>> +@section Memory Allocation Tunables
>
> Some @cindex entries here would be good, particularly tying together
> malloc + tunables; e.g.:
>
> @cindex memory allocation tunables
> @cindex malloc, tunables
> @cindex tunables, malloc
>
OK
>> +
>> +@deftp Namespace glibc.malloc
>> +Memory allocation behaviour can be modified by setting any of the
>> +following tunables in the @code{malloc} namespace.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.check
>> +This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is
>> +identical in features.
>> +
>> +Setting this tunable enables a special (less efficient) memory allocator for
>> +the malloc family of functions that is designed to be tolerant against simple
>> +errors such as double calls of free with the same argument, or overruns of a
>> +single byte (off-by-one bugs). Not all such errors can be protected against,
>> +however, and memory leaks can result. The following list describes the values
>> +that this tunable can take and the effect they have on malloc functionality.
>> +
>> +@itemize @bullet
>> +@item @code{0} Disable all error reporting. The alternate allocator is
>> +selected and heap corruption detection is in place, but any such errors
>> +detected are ignored. This is currently a supported use, but is not
>> +recommended.
>> +@item @code{1} Report errors. The alternate allocator is selected and heap
>> +corruption, if detected, is reported as diagnostic messages to @code{stderr}
>> +and the program continues execution.
>> +@item @code{2} Abort on errors. The alternate allocator is selected and if
>> +heap corruption is detected, the program is ended immediately by calling
>> +@code{abort}.
>> +@item @code{3} Fully enabled. The alternate allocator is selected and is fully
>> +functional. That is, if heap corruption is detected, a verbose diagnostic
>> +message is printed to @code{stderr} and the program is ended by calling
>> +@code{abort}.
>> +@end itemize
>> +
>> +Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it
>> +diverges from normal program behavior by writing to @code{stderr}, which could
>> +by exploited in SUID and SGID binaries. Therefore, @code{glibc.malloc.check}
>> +is disabled by default for SUID and SGID binaries. This can be enabled again
>> +by the system administrator by adding a file @file{/etc/suid-debug}; the
>> +content of the file could be anything or even empty.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.top_pad
>> +This tunable supersedes the @code{MALLOC_TOP_PAD_} environment variable and is
>> +identical in features.
>> +
>> +This tunable determines the amount of extra memory to obtain from the system
>
> "amount of extra memory in bytes..."; I think it would be nice if the
> first sentence mentioned the units.
>
OK
>> +when any of the arenas need to be extended. It also specifies the number of
>> +bytes to retain when shrinking any of the arenas. This provides the necessary
>> +hysteresis in heap size such that excessive amounts of system calls can be
>> +avoided.
>> +
>> +The default value of this tunable is @samp{0}.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.perturb
>> +This tunable supersedes the @code{MALLOC_PERTURB_} environment variable and is
>> +identical in features.
>> +
>> +If set to a non-zero value, memory blocks are initialized with values depending
>> +on some low order bits of this tunable when they are allocated (except when
>> +allocated by calloc) and freed. This can be used to debug the use of
>> +uninitialized or freed heap memory. Note that this option does not guarantee
>> +that the freed block will have any specific values. It only guarantees that the
>> +content the block had before it was freed will be overwritten.
>
> This seems non-deterministic and leaves me wondering which low-order
> bits, and how to select my values so I know what gets written.
>
The set value is xor'd with 0xff, so it is not the exact same byte value
that gets overwritten. Exposing the value (0xff) inhibits us from
changing the byte sequence that gets written in to overwrite (in case
there is an exploit based on the value of the perturb_byte sequence in
future) which is why the value is purposely vaguely defined.
>> +
>> +The default value of this tunable is @samp{0}.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.mmap_threshold
>> +This tunable supersedes the @code{MALLOC_MMAP_THRESHOLD_} environment variable
>> +and is identical in features.
>> +
>> +When this tunable is set, all chunks larger than this value are allocated
>> +outside the normal heap, using the @code{mmap} system call. This way it is
>> +guaranteed that the memory for these chunks can be returned to the system on
>> +@code{free}. Note that requests smaller than this threshold might still be
>> +allocated via @code{mmap}.
>
> Units here, too.
>
OK
>> +
>> +If this tunable is not set, the default value is set as 128 KB and the
>> +threshold is adjusted dynamically to suit the allocation patterns of the
>> +program. If the tunable is set, the dynamic adjustment is disabled and the
>> +value is set as static.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.trim_threshold
>> +This tunable supersedes the @code{MALLOC_TRIM_THRESHOLD_} environment variable
>> +and is identical in features.
>> +
>> +The value of this tunable is the minimum size (in bytes) of the top-most,
>> +releasable chunk in an arena that will trigger a system call in order to return
>> +memory to the system from that arena.
>> +
>> +If this tunable is not set, the default value is set as 128 KB and the
>
> set to
>
> I thought this was in bytes! ;)
>
:)
>> +threshold is adjusted dynamically to suit the allocation patterns of the
>> +program. If the tunable is set, the dynamic adjustment is disabled and the
>> +value is set as static.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.mmap_max
>> +This tunable supersedes the @code{MALLOC_MMAP_MAX_} environment variable and is
>> +identical in features.
>> +
>> +The value of this tunable is maximum number of chunks to allocate with
>> +@code{mmap}. Setting this to zero disables all use of @code{mmap}.
>> +
>> +The default value of this tunable is @samp{65536}.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.arena_test
>> +This tunable supersedes the @code{MALLOC_ARENA_TEST} environment variable and is
>> +identical in features.
>> +
>> +The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that
>> +can be created before the test on the limit to the number of arenas is
>> +conducted. The value is ignored if @code{glibc.malloc.arena_max} is set.
>> +
>> +The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit
>> +systems.
>> +@end deftp
>> +
>> +@deftp Tunable glibc.malloc.arena_max
>> +This tunable supersedes the @code{MALLOC_ARENA_MAX} environment variable and is
>> +identical in features.
>> +
>> +This tunable sets the number of arenas to use in a process regardless of the
>> +number of cores in the system.
>> +
>> +The default value of this tunable is @code{0}, meaning that the limit on the
>> +number of arenas is determined by the number of CPU cores online. For 32-bit
>> +systems the limit is twice the number of cores online and on 64-bit systems, it
>> +is 8 times the number of cores online.
>> +@end deftp
>
> Whitespace issues throughout; noticed a lot of single-space separated
> sentences. Not sure that it matters in the rendered output, but it's
> nice for source readability.
>
It does not matter for rendered output, but I'll fix them up anyway.
> I like how you made this its own chapter. I think it's a good start.
Thanks, I'll fix these up locally and wait for reviews of other patches.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PING 2][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (4 preceding siblings ...)
2016-11-21 2:58 ` [ping][PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
@ 2016-11-24 13:31 ` Siddhesh Poyarekar
2016-11-28 11:52 ` [PING 3][PATCHv6 " Siddhesh Poyarekar
` (4 subsequent siblings)
10 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-24 13:31 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PING 3][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (5 preceding siblings ...)
2016-11-24 13:31 ` [PING 2][PATCHv6 " Siddhesh Poyarekar
@ 2016-11-28 11:52 ` Siddhesh Poyarekar
2016-12-05 12:08 ` [PING 4][PATCHv6 " Siddhesh Poyarekar
` (3 subsequent siblings)
10 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-11-28 11:52 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PING 4][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (6 preceding siblings ...)
2016-11-28 11:52 ` [PING 3][PATCHv6 " Siddhesh Poyarekar
@ 2016-12-05 12:08 ` Siddhesh Poyarekar
2016-12-05 15:10 ` Steven Munroe
2016-12-12 16:20 ` [PING 5][PATCHv6 " Siddhesh Poyarekar
` (2 subsequent siblings)
10 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-05 12:08 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PING 4][PATCHv6 0/4] glibc tunables
2016-12-05 12:08 ` [PING 4][PATCHv6 " Siddhesh Poyarekar
@ 2016-12-05 15:10 ` Steven Munroe
0 siblings, 0 replies; 26+ messages in thread
From: Steven Munroe @ 2016-12-05 15:10 UTC (permalink / raw)
To: Siddhesh Poyarekar; +Cc: libc-alpha, carlos, adhemerval.zanella
On Mon, 2016-12-05 at 17:38 +0530, Siddhesh Poyarekar wrote:
> Ping!
This is important and long over due. We need to this includes in 2.25.
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PING 5][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (7 preceding siblings ...)
2016-12-05 12:08 ` [PING 4][PATCHv6 " Siddhesh Poyarekar
@ 2016-12-12 16:20 ` Siddhesh Poyarekar
2016-12-26 14:29 ` [PING 6][PATCHv6 " Siddhesh Poyarekar
2016-12-27 9:49 ` [PATCHv6 " Florian Weimer
10 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-12 16:20 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* [PING 6][PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (8 preceding siblings ...)
2016-12-12 16:20 ` [PING 5][PATCHv6 " Siddhesh Poyarekar
@ 2016-12-26 14:29 ` Siddhesh Poyarekar
2016-12-27 9:49 ` [PATCHv6 " Florian Weimer
10 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-26 14:29 UTC (permalink / raw)
To: libc-alpha; +Cc: carlos, adhemerval.zanella, Florian Weimer
Ping!
On Wednesday 16 November 2016 02:05 PM, Siddhesh Poyarekar wrote:
> Hi,
>
> Here is iteration 6 of the tunables patchset, now with two patches palmed off
> to Adhemerval to worry about! Here are changes from the previous patchset:
>
> - I discovered some implicit includes that broke aarch64 - shame on me for
> missing them.
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
> - The patchset depends on Adhemerval's __access_noerrno patch[1] fixed up with
> my comments on the patch.
>
> The branch siddhesh/tunables has these patches and also Adhemerval's fixed up
> patch. If you're watching this, please also review Adhemerval's
> __access_noerrno patch and help get that through.
>
> Siddhesh
>
> [1] https://sourceware.org/ml/libc-alpha/2016-11/msg00416.html
>
> Siddhesh Poyarekar (4):
> Add framework for tunables
> Initialize tunable list with the GLIBC_TUNABLES environment variable
> Enhance --enable-tunables to select tunables frontend at build time
> User manual documentation for tunables
>
> INSTALL | 18 ++
> Makeconfig | 16 ++
> README.tunables | 84 ++++++
> config.h.in | 3 +
> config.make.in | 1 +
> configure | 17 ++
> configure.ac | 10 +
> csu/init-first.c | 2 -
> csu/libc-start.c | 12 +
> elf/Makefile | 7 +
> elf/Versions | 3 +
> elf/dl-support.c | 2 +
> elf/dl-sysdep.c | 8 +
> elf/dl-tunable-types.h | 46 +++
> elf/dl-tunables.c | 447 +++++++++++++++++++++++++++++
> elf/dl-tunables.h | 82 ++++++
> elf/dl-tunables.list | 69 +++++
> elf/rtld.c | 2 +
> malloc/Makefile | 6 +
> malloc/arena.c | 54 ++++
> malloc/tst-malloc-usable-static-tunables.c | 1 +
> malloc/tst-malloc-usable-static.c | 1 +
> malloc/tst-malloc-usable-tunables.c | 1 +
> manual/Makefile | 3 +-
> manual/install.texi | 21 ++
> manual/probes.texi | 2 +-
> manual/tunables.texi | 185 ++++++++++++
> scripts/gen-tunables.awk | 157 ++++++++++
> sysdeps/mach/hurd/dl-sysdep.c | 8 +
> 29 files changed, 1264 insertions(+), 4 deletions(-)
> create mode 100644 README.tunables
> create mode 100644 elf/dl-tunable-types.h
> create mode 100644 elf/dl-tunables.c
> create mode 100644 elf/dl-tunables.h
> create mode 100644 elf/dl-tunables.list
> create mode 100644 malloc/tst-malloc-usable-static-tunables.c
> create mode 100644 malloc/tst-malloc-usable-static.c
> create mode 100644 malloc/tst-malloc-usable-tunables.c
> create mode 100644 manual/tunables.texi
> create mode 100644 scripts/gen-tunables.awk
>
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCHv6 0/4] glibc tunables
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
` (9 preceding siblings ...)
2016-12-26 14:29 ` [PING 6][PATCHv6 " Siddhesh Poyarekar
@ 2016-12-27 9:49 ` Florian Weimer
10 siblings, 0 replies; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 9:49 UTC (permalink / raw)
To: Siddhesh Poyarekar, libc-alpha; +Cc: carlos, adhemerval.zanella
On 11/16/2016 09:35 AM, Siddhesh Poyarekar wrote:
> - I realized that __tunables_init used strncmp and strlen. They're built into
> ld.so too, but they are likely tunables targets and I don't want them to
> break unintentionally in future, so off they go
In statically linked binaries, tunables_strdup calls __mmap before the
TCB is initialized. This is not supported.
With --enable-stack-protector=all, __getpagesize receive stack protector
instrumentation, and binaries crash because it is called in
tunables_strdup. This call is unnecessary because the length argument
to mmap does not have to be a multiple of the page size.
However, even if __getpagesize is removed, the __mmap problem will
persist. The existing TCB initialization code uses sbrk instead of
mmap, but has the same issue.
It might be possible to avoid copying the string at all if you do not
require that string parameters are NULL-terminated, and instead store
the length explicitly.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-11-16 8:35 ` [PATCH 1/4] Add framework for tunables Siddhesh Poyarekar
@ 2016-12-27 11:08 ` Florian Weimer
2016-12-27 20:59 ` Siddhesh Poyarekar
2016-12-30 13:28 ` Joseph Myers
0 siblings, 2 replies; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 11:08 UTC (permalink / raw)
To: Siddhesh Poyarekar, libc-alpha; +Cc: carlos, adhemerval.zanella
On 11/16/2016 09:35 AM, Siddhesh Poyarekar wrote:
> diff --git a/README.tunables b/README.tunables
> +The tunable framework allows modules within glibc to register variables that
> +may be tweaked through an environment variable or an API call. It aims to
This seems outdated, change with an API call is no longer supported.
> +enforce a strict namespace rule to bring consistency to naming of these tunable
> +environment variables across the project. This document is a guide for glibc
> +developers to add tunables to the framework.
> +
> +ADDING A NEW TUNABLE
> +--------------------
> +
> +The TOP_NAMESPACE is defined by default as 'glibc' and it may be overridden in
“The TOP_NAMESPACE macro”?
It's not clear based on this description if distributions are supposed
to change the default definition for glibc as a whole (I don't think
so), or where they are supposed to override the macro.
> diff --git a/csu/libc-start.c b/csu/libc-start.c
> index 99c040a..4e5cafa 100644
> --- a/csu/libc-start.c
> +++ b/csu/libc-start.c
> @@ -21,6 +21,11 @@
> #include <unistd.h>
> #include <ldsodefs.h>
> #include <exit-thread.h>
> +#include <libc-internal.h>
> +
> +#if HAVE_TUNABLES
> +# include <elf/dl-tunables.h>
> +#endif
I'd prefer if you could cut down the number of HAVE_TUNABLES conditionals.
> + /* Initialize very early so that tunables can use it. */
> + __libc_init_secure ();
> +
> +#if HAVE_TUNABLES
> + __tunables_init (__environ);
> +#endif
For !HAVE_TUNABLES, __tunables_init could be defined an empty inline
function, so that no code is generated for it.
> diff --git a/elf/dl-tunable-types.h b/elf/dl-tunable-types.h
> +#ifndef _TUNABLE_TYPES_H_
> +# define _TUNABLE_TYPES_H_
No indentation here.
> diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
> new file mode 100644
> index 0000000..91340b6
> --- /dev/null
> +++ b/elf/dl-tunables.c
> @@ -0,0 +1,310 @@
> +/* The tunable framework. See the README to know how to use the tunable in
Should be “README.tunables”.
Parts of this file should probably go into csu/ because the code runs so
early during library startup (see my other comments).
> +static int
> +tunables_unsetenv (char **ep, const char *name)
This is copied from elf/dl-environ.c, it seems. Is it possible to avoid
creating this copy?
Calling either form of unsetenv is problematic because it may make the
auxiliary vector unreachable on !LIBC_START_MAIN_AUXVEC_ARG platforms
(which seems to include the majority, so perhaps this is a non-issue).
> +/* If the value does not validate, don't bother initializing the internal type
> + and also take care to clear the recorded string value in STRVAL. */
> +#define RETURN_IF_INVALID_RANGE(__cur, __val) \
> +({ \
> + __typeof ((__cur)->type.min) min = (__cur)->type.min; \
> + __typeof ((__cur)->type.max) max = (__cur)->type.max; \
> + if (min != max && ((__val) < min || (__val) > max)) \
> + return; \
> +})
Is it really necessary to turn this into a macro instead of an (inline)
function? The return statement could very well be located in the
caller, IMHO.
> +/* Disable a tunable if it is set. */
> +static void
> +disable_tunable (tunable_id_t id, char **envp)
> +{
> + const char *env_alias = tunable_list[id].env_alias;
> +
> + if (env_alias)
> + tunables_unsetenv (envp, tunable_list[id].env_alias);
> +}
env_alias != NULL.
> + default:
> + __builtin_unreachable ();
Should this rather be __bultin_trap?
> diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
> +/* A tunable. */
> +struct _tunable
> +{
> + const char *env_alias; /* The compatibility environment
> + variable name. */
Maybe store the variable name as well, as to
> +/* Same as TUNABLE_SET_VAL, but also set the callback function to __CB and call
> + it. */
> +# define TUNABLE_SET_VAL_WITH_CALLBACK(__id,__val,__cb) \
Comment seems misleading, the callback function isn't really set.
> diff --git a/elf/rtld.c b/elf/rtld.c
> index 647661c..eb211a2 100644
> --- a/elf/rtld.c
> +++ b/elf/rtld.c
> @@ -2527,11 +2527,13 @@ process_envvars (enum mode *modep)
> }
> while (*nextp != '\0');
>
> +#if !HAVE_TUNABLES
> if (__access ("/etc/suid-debug", F_OK) != 0)
> {
> unsetenv ("MALLOC_CHECK_");
> GLRO(dl_debug_mask) = 0;
> }
> +#endif
I believe this introduces a security vulnerability. You must keep the
setting of dl_debug_mask.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable
2016-11-16 8:35 ` [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable Siddhesh Poyarekar
@ 2016-12-27 11:41 ` Florian Weimer
2016-12-27 21:35 ` Siddhesh Poyarekar
0 siblings, 1 reply; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 11:41 UTC (permalink / raw)
To: Siddhesh Poyarekar, libc-alpha; +Cc: carlos, adhemerval.zanella
On 11/16/2016 09:35 AM, Siddhesh Poyarekar wrote:
> diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
> +static char *
> +tunables_strdup (const char *in)
> +{
> + size_t i = 0;
> +
> + while (in[i++]);
Please use an explicit comparison against '\0'.
> + char *out = __mmap (NULL, ALIGN_UP (i, __getpagesize ()),
> + PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1,
> + 0);
> +
> + if (out == MAP_FAILED)
> + return NULL;
See my initial mail. errno and __getpagesize are not available at this
point.
> +static size_t
> +min_strlen (const char *s)
> +{
> + size_t i = 0;
> + while (*s++ != '\0')
> + i++;
> +
> + return i;
> +}
Is there anything which ensures that GCC does not replace this
implementation with strlen?
> /* Disable a tunable if it is set. */
> static void
> disable_tunable (tunable_id_t id, char **envp)
> @@ -216,6 +318,23 @@ disable_tunable (tunable_id_t id, char **envp)
>
> if (env_alias)
> tunables_unsetenv (envp, tunable_list[id].env_alias);
> +
> + char *tunable = getenv (GLIBC_TUNABLES);
> + const char *cmp = tunable_list[id].name;
> + const size_t len = min_strlen (cmp);
> +
> + while (tunable && *tunable != '\0' && *tunable != ':')
> + {
> + if (is_name (tunable, cmp))
> + {
> + tunable += len;
> + /* Overwrite the = and the value with colons. */
> + while (*tunable != '\0' && *tunable != ':')
> + *tunable++ = ':';
> + break;
> + }
> + tunable++;
> + }
> }
This assumes that the process environment is not mapped read-only. I'm
not sure if this is guaranteed by the ABI.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time
2016-11-16 8:35 ` [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time Siddhesh Poyarekar
@ 2016-12-27 11:44 ` Florian Weimer
2016-12-27 21:10 ` Siddhesh Poyarekar
0 siblings, 1 reply; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 11:44 UTC (permalink / raw)
To: Siddhesh Poyarekar, libc-alpha; +Cc: carlos, adhemerval.zanella
On 11/16/2016 09:35 AM, Siddhesh Poyarekar wrote:
> At the GNU Tools Cauldron 2016, the state of the current tunables
> patchset was considered OK with the addition of a way to select the
> frontend to be used for the tunables. That is, to avoid being locked
> in to one type of frontend initially, it should be possible to build
> tunables with a different frontend with something as simple as a
> configure switch.
>
> To that effect, this patch enhances the --enable-tunables option to
> accept more values than just 'yes' or 'no'. The current frontend (and
> default when enable-tunables is 'yes') is called 'valstring', to
> select the frontend where a single environment variable is set to a
> colon-separated value string. More such frontends can be added in
> future.
I'm not sure if this patch makes sense at this point because without
alternative frontends, it's impossible to tell if the current
initialization sequence is sufficient.
For example, if we want to load a tunable configuration with very low
cost, we could use ldconfig to pre-parse a configuration file and put it
into /etc/ld.so.cache. But we won't know if the current approach works
until this is actually implemented.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-12-27 11:08 ` Florian Weimer
@ 2016-12-27 20:59 ` Siddhesh Poyarekar
2016-12-27 21:12 ` Florian Weimer
2016-12-30 13:28 ` Joseph Myers
1 sibling, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-27 20:59 UTC (permalink / raw)
To: Florian Weimer, libc-alpha; +Cc: carlos, adhemerval.zanella
On Tuesday 27 December 2016 04:38 PM, Florian Weimer wrote:
> On 11/16/2016 09:35 AM, Siddhesh Poyarekar wrote:
>> diff --git a/README.tunables b/README.tunables
>
>> +The tunable framework allows modules within glibc to register
>> variables that
>> +may be tweaked through an environment variable or an API call. It
>> aims to
>
> This seems outdated, change with an API call is no longer supported.
Yeah, that's an old relic. Fixed
>> +enforce a strict namespace rule to bring consistency to naming of
>> these tunable
>> +environment variables across the project. This document is a guide
>> for glibc
>> +developers to add tunables to the framework.
>> +
>> +ADDING A NEW TUNABLE
>> +--------------------
>> +
>> +The TOP_NAMESPACE is defined by default as 'glibc' and it may be
>> overridden in
>
> “The TOP_NAMESPACE macro”?
Fixed.
>
> It's not clear based on this description if distributions are supposed
> to change the default definition for glibc as a whole (I don't think
> so), or where they are supposed to override the macro.
>
I'll redo the description a bit.
>> diff --git a/csu/libc-start.c b/csu/libc-start.c
>> index 99c040a..4e5cafa 100644
>> --- a/csu/libc-start.c
>> +++ b/csu/libc-start.c
>> @@ -21,6 +21,11 @@
>> #include <unistd.h>
>> #include <ldsodefs.h>
>> #include <exit-thread.h>
>> +#include <libc-internal.h>
>> +
>> +#if HAVE_TUNABLES
>> +# include <elf/dl-tunables.h>
>> +#endif
>
> I'd prefer if you could cut down the number of HAVE_TUNABLES conditionals.
>
>> + /* Initialize very early so that tunables can use it. */
>> + __libc_init_secure ();
>> +
>> +#if HAVE_TUNABLES
>> + __tunables_init (__environ);
>> +#endif
>
> For !HAVE_TUNABLES, __tunables_init could be defined an empty inline
> function, so that no code is generated for it.
>
OK.
>> diff --git a/elf/dl-tunable-types.h b/elf/dl-tunable-types.h
>
>> +#ifndef _TUNABLE_TYPES_H_
>> +# define _TUNABLE_TYPES_H_
>
> No indentation here.
>
>> diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
>> new file mode 100644
>> index 0000000..91340b6
>> --- /dev/null
>> +++ b/elf/dl-tunables.c
>> @@ -0,0 +1,310 @@
>> +/* The tunable framework. See the README to know how to use the
>> tunable in
>
> Should be “README.tunables”.
OK.
> Parts of this file should probably go into csu/ because the code runs so
> early during library startup (see my other comments).
OK, I'll need to figure out how to split things up.
>> +static int
>> +tunables_unsetenv (char **ep, const char *name)
>
> This is copied from elf/dl-environ.c, it seems. Is it possible to avoid
> creating this copy?
I'm going to eventually drop the dl-environ.c copy by moving the LD_*
variables to tunables.
>
> Calling either form of unsetenv is problematic because it may make the
> auxiliary vector unreachable on !LIBC_START_MAIN_AUXVEC_ARG platforms
> (which seems to include the majority, so perhaps this is a non-issue).
I'm not sure what the problem is here. Either way, it doesn't seem like
something the patch introduces, is it?
>> +/* If the value does not validate, don't bother initializing the
>> internal type
>> + and also take care to clear the recorded string value in STRVAL. */
>> +#define RETURN_IF_INVALID_RANGE(__cur, __val) \
>> +({ \
>> + __typeof ((__cur)->type.min) min =
>> (__cur)->type.min; \
>> + __typeof ((__cur)->type.max) max =
>> (__cur)->type.max; \
>> + if (min != max && ((__val) < min || (__val) > max)) \
>> + return; \
>> +})
>
> Is it really necessary to turn this into a macro instead of an (inline)
> function? The return statement could very well be located in the
> caller, IMHO.
I rewrote the function to explicitly check for bounds based on type.
>
>> +/* Disable a tunable if it is set. */
>> +static void
>> +disable_tunable (tunable_id_t id, char **envp)
>> +{
>> + const char *env_alias = tunable_list[id].env_alias;
>> +
>> + if (env_alias)
>> + tunables_unsetenv (envp, tunable_list[id].env_alias);
>> +}
>
> env_alias != NULL.
>
>> + default:
>> + __builtin_unreachable ();
>
> Should this rather be __bultin_trap?
OK.
>
>> diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
>
>> +/* A tunable. */
>> +struct _tunable
>> +{
>
>> + const char *env_alias; /* The compatibility environment
>> + variable name. */
>
> Maybe store the variable name as well, as to
I don't understand this, can you please elaborate?
>
>> +/* Same as TUNABLE_SET_VAL, but also set the callback function to
>> __CB and call
>> + it. */
>> +# define TUNABLE_SET_VAL_WITH_CALLBACK(__id,__val,__cb) \
>
> Comment seems misleading, the callback function isn't really set.
It used to be until I changed that. I'll fix the comment.
>
>> diff --git a/elf/rtld.c b/elf/rtld.c
>> index 647661c..eb211a2 100644
>> --- a/elf/rtld.c
>> +++ b/elf/rtld.c
>> @@ -2527,11 +2527,13 @@ process_envvars (enum mode *modep)
>> }
>> while (*nextp != '\0');
>>
>> +#if !HAVE_TUNABLES
>> if (__access ("/etc/suid-debug", F_OK) != 0)
>> {
>> unsetenv ("MALLOC_CHECK_");
>> GLRO(dl_debug_mask) = 0;
>> }
>> +#endif
>
> I believe this introduces a security vulnerability. You must keep the
> setting of dl_debug_mask.
I'll add it back in #ifdef SHARED in disable_tunable.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time
2016-12-27 11:44 ` Florian Weimer
@ 2016-12-27 21:10 ` Siddhesh Poyarekar
2016-12-27 21:16 ` Florian Weimer
0 siblings, 1 reply; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-27 21:10 UTC (permalink / raw)
To: Florian Weimer, libc-alpha; +Cc: carlos, adhemerval.zanella
On Tuesday 27 December 2016 05:14 PM, Florian Weimer wrote:
> I'm not sure if this patch makes sense at this point because without
> alternative frontends, it's impossible to tell if the current
> initialization sequence is sufficient.
>
> For example, if we want to load a tunable configuration with very low
> cost, we could use ldconfig to pre-parse a configuration file and put it
> into /etc/ld.so.cache. But we won't know if the current approach works
> until this is actually implemented.
The option --enable-tunables=... doesn't actually do anything beyond
explicitly naming the current tunables frontend implementation. To
write a different frontend, one would not only have to add a new value
to the --enable-tunables=... option but also tweak the current
initialization sequence to suit its frontend without hurting existing
frontends.
I did not intend to make a framework to allow plugging in of different
frontends directly because from the discussions at Cauldron I concluded
that the primary reason for such a requirement was more to loosen the
guarantee of availability of a specific frontend than to allow
co-existence of different frontends at the same time.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-12-27 20:59 ` Siddhesh Poyarekar
@ 2016-12-27 21:12 ` Florian Weimer
2016-12-27 21:25 ` Siddhesh Poyarekar
0 siblings, 1 reply; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 21:12 UTC (permalink / raw)
To: siddhesh, libc-alpha; +Cc: carlos, adhemerval.zanella
On 12/27/2016 09:59 PM, Siddhesh Poyarekar wrote:
>>> +static int
>>> +tunables_unsetenv (char **ep, const char *name)
>>
>> This is copied from elf/dl-environ.c, it seems. Is it possible to avoid
>> creating this copy?
>
> I'm going to eventually drop the dl-environ.c copy by moving the LD_*
> variables to tunables.
This is fine. Perhaps add a comment.
>> Calling either form of unsetenv is problematic because it may make the
>> auxiliary vector unreachable on !LIBC_START_MAIN_AUXVEC_ARG platforms
>> (which seems to include the majority, so perhaps this is a non-issue).
>
> I'm not sure what the problem is here.
If auxv isn't passed explicitly, the vector starts after the last NULL
pointer in the envp array. If you put in additional NULL pointers there
to remove element, that's not going to work anymore.
> Either way, it doesn't seem like
> something the patch introduces, is it?
Probably not, but I don't have a complete picture in my head of the
startup code.
>>> diff --git a/elf/dl-tunables.h b/elf/dl-tunables.h
>>
>>> +/* A tunable. */
>>> +struct _tunable
>>> +{
>>
>>> + const char *env_alias; /* The compatibility environment
>>> + variable name. */
>>
>> Maybe store the variable name as well, as to
>
> I don't understand this, can you please elaborate?
I meant to write “variable name length”, so that it's possible to
optimize the comparison somewhat. But it's likely better to keep the
complexity down for now, so please disregard this suggestion.
>> I believe this introduces a security vulnerability. You must keep the
>> setting of dl_debug_mask.
>
> I'll add it back in #ifdef SHARED in disable_tunable.
Are you sure this is sufficient? You need to make sure that the value
is zeroed out again after it has been populated from the environment and
before it is used in any way by the dynamic linker.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time
2016-12-27 21:10 ` Siddhesh Poyarekar
@ 2016-12-27 21:16 ` Florian Weimer
0 siblings, 0 replies; 26+ messages in thread
From: Florian Weimer @ 2016-12-27 21:16 UTC (permalink / raw)
To: siddhesh, libc-alpha; +Cc: carlos, adhemerval.zanella
On 12/27/2016 10:10 PM, Siddhesh Poyarekar wrote:
> On Tuesday 27 December 2016 05:14 PM, Florian Weimer wrote:
>> I'm not sure if this patch makes sense at this point because without
>> alternative frontends, it's impossible to tell if the current
>> initialization sequence is sufficient.
>>
>> For example, if we want to load a tunable configuration with very low
>> cost, we could use ldconfig to pre-parse a configuration file and put it
>> into /etc/ld.so.cache. But we won't know if the current approach works
>> until this is actually implemented.
>
> The option --enable-tunables=... doesn't actually do anything beyond
> explicitly naming the current tunables frontend implementation. To
> write a different frontend, one would not only have to add a new value
> to the --enable-tunables=... option but also tweak the current
> initialization sequence to suit its frontend without hurting existing
> frontends.
Okay. I don't know what was decided at the Cauldron because it wasn't
there. The patch doesn't look burdensome going forward, so maybe we can
keep it if it makes someone happy.
Thanks,
Florian
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-12-27 21:12 ` Florian Weimer
@ 2016-12-27 21:25 ` Siddhesh Poyarekar
0 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-27 21:25 UTC (permalink / raw)
To: Florian Weimer, libc-alpha; +Cc: carlos, adhemerval.zanella
On Wednesday 28 December 2016 02:41 AM, Florian Weimer wrote:
> Are you sure this is sufficient? You need to make sure that the value
> is zeroed out again after it has been populated from the environment and
> before it is used in any way by the dynamic linker.
Right, it isn't. I guess I'll be better off only avoiding the unsetenv
in rtld.c for now when !HAVE_TUNABLES and clearing dl_debug_mask all the
time.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable
2016-12-27 11:41 ` Florian Weimer
@ 2016-12-27 21:35 ` Siddhesh Poyarekar
0 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-27 21:35 UTC (permalink / raw)
To: Florian Weimer, libc-alpha; +Cc: carlos, adhemerval.zanella
On Tuesday 27 December 2016 05:11 PM, Florian Weimer wrote:
> See my initial mail. errno and __getpagesize are not available at this
> point.
Ugh, right. I'll try to come up with something that does not need an
additional allocation.
>> +static size_t
>> +min_strlen (const char *s)
>> +{
>> + size_t i = 0;
>> + while (*s++ != '\0')
>> + i++;
>> +
>> + return i;
>> +}
>
> Is there anything which ensures that GCC does not replace this
> implementation with strlen?
I think I'm missing a gcc flag here to ensure that. I'll find out and
add it.
>> /* Disable a tunable if it is set. */
>> static void
>> disable_tunable (tunable_id_t id, char **envp)
>> @@ -216,6 +318,23 @@ disable_tunable (tunable_id_t id, char **envp)
>>
>> if (env_alias)
>> tunables_unsetenv (envp, tunable_list[id].env_alias);
>> +
>> + char *tunable = getenv (GLIBC_TUNABLES);
>> + const char *cmp = tunable_list[id].name;
>> + const size_t len = min_strlen (cmp);
>> +
>> + while (tunable && *tunable != '\0' && *tunable != ':')
>> + {
>> + if (is_name (tunable, cmp))
>> + {
>> + tunable += len;
>> + /* Overwrite the = and the value with colons. */
>> + while (*tunable != '\0' && *tunable != ':')
>> + *tunable++ = ':';
>> + break;
>> + }
>> + tunable++;
>> + }
>> }
>
> This assumes that the process environment is not mapped read-only. I'm
> not sure if this is guaranteed by the ABI.
Barring tunables with string values this might be easy to work around.
Let me give this a shot too.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-12-27 11:08 ` Florian Weimer
2016-12-27 20:59 ` Siddhesh Poyarekar
@ 2016-12-30 13:28 ` Joseph Myers
2016-12-31 13:48 ` Siddhesh Poyarekar
1 sibling, 1 reply; 26+ messages in thread
From: Joseph Myers @ 2016-12-30 13:28 UTC (permalink / raw)
To: Florian Weimer; +Cc: Siddhesh Poyarekar, libc-alpha, carlos, adhemerval.zanella
On Tue, 27 Dec 2016, Florian Weimer wrote:
> > + default:
> > + __builtin_unreachable ();
>
> Should this rather be __bultin_trap?
Note that __bultin_trap generates a call to abort on some platforms. It
can only be used in places where such a call is OK; we've had problems
with the isolate-erroneous-paths optimization generating such calls. So
make sure anything using __bultin_trap builds OK for SH with
build-many-glibcs.py, for example.
--
Joseph S. Myers
joseph@codesourcery.com
^ permalink raw reply [flat|nested] 26+ messages in thread
* Re: [PATCH 1/4] Add framework for tunables
2016-12-30 13:28 ` Joseph Myers
@ 2016-12-31 13:48 ` Siddhesh Poyarekar
0 siblings, 0 replies; 26+ messages in thread
From: Siddhesh Poyarekar @ 2016-12-31 13:48 UTC (permalink / raw)
To: Joseph Myers, Florian Weimer; +Cc: libc-alpha, carlos, adhemerval.zanella
On Friday 30 December 2016 06:57 PM, Joseph Myers wrote:
> Note that __bultin_trap generates a call to abort on some platforms. It
> can only be used in places where such a call is OK; we've had problems
> with the isolate-erroneous-paths optimization generating such calls. So
> make sure anything using __bultin_trap builds OK for SH with
> build-many-glibcs.py, for example.
Oh, then I think I'll stick to __builtin_unreachable for now and then
maybe post an addon patch to use __builtin_trap once I have verified all
target builds. I'm setting up the tree for build-many-glibcs right now.
Siddhesh
^ permalink raw reply [flat|nested] 26+ messages in thread
end of thread, other threads:[~2016-12-31 13:48 UTC | newest]
Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-16 8:35 [PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 4/4] User manual documentation for tunables Siddhesh Poyarekar
2016-11-16 13:49 ` Rical Jasan
2016-11-21 3:07 ` Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 2/4] Initialize tunable list with the GLIBC_TUNABLES environment variable Siddhesh Poyarekar
2016-12-27 11:41 ` Florian Weimer
2016-12-27 21:35 ` Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 1/4] Add framework for tunables Siddhesh Poyarekar
2016-12-27 11:08 ` Florian Weimer
2016-12-27 20:59 ` Siddhesh Poyarekar
2016-12-27 21:12 ` Florian Weimer
2016-12-27 21:25 ` Siddhesh Poyarekar
2016-12-30 13:28 ` Joseph Myers
2016-12-31 13:48 ` Siddhesh Poyarekar
2016-11-16 8:35 ` [PATCH 3/4] Enhance --enable-tunables to select tunables frontend at build time Siddhesh Poyarekar
2016-12-27 11:44 ` Florian Weimer
2016-12-27 21:10 ` Siddhesh Poyarekar
2016-12-27 21:16 ` Florian Weimer
2016-11-21 2:58 ` [ping][PATCHv6 0/4] glibc tunables Siddhesh Poyarekar
2016-11-24 13:31 ` [PING 2][PATCHv6 " Siddhesh Poyarekar
2016-11-28 11:52 ` [PING 3][PATCHv6 " Siddhesh Poyarekar
2016-12-05 12:08 ` [PING 4][PATCHv6 " Siddhesh Poyarekar
2016-12-05 15:10 ` Steven Munroe
2016-12-12 16:20 ` [PING 5][PATCHv6 " Siddhesh Poyarekar
2016-12-26 14:29 ` [PING 6][PATCHv6 " Siddhesh Poyarekar
2016-12-27 9:49 ` [PATCHv6 " Florian Weimer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).