From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 93527 invoked by alias); 16 Nov 2016 13:49:16 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Received: (qmail 93510 invoked by uid 89); 16 Nov 2016 13:49:16 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-1.0 required=5.0 tests=AWL,BAYES_80,RCVD_IN_DNSWL_NONE,RP_MATCHES_RCVD,SPF_PASS autolearn=no version=3.3.2 spammy=conducted, arena, suit, @samp X-Spam-User: qpsmtpd, 2 recipients X-HELO: mail.pacific.net Subject: Re: [PATCH 4/4] User manual documentation for tunables To: Siddhesh Poyarekar , libc-alpha@sourceware.org References: <1479285306-11684-5-git-send-email-siddhesh@sourceware.org> Cc: carlos@redhat.com, adhemerval.zanella@linaro.org From: Rical Jasan Message-ID: <8f128e69-e68e-9cdf-577e-64dffc7297c9@pacific.net> Date: Wed, 16 Nov 2016 13:49:00 -0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-Version: 1.0 In-Reply-To: <1479285306-11684-5-git-send-email-siddhesh@sourceware.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Null-Tag: 3f44d8cd42cf0c9649bf4a8f5a359fdd X-SW-Source: 2016-11/txt/msg00548.txt.bz2 On 11/16/2016 12:35 AM, Siddhesh Poyarekar wrote: > Create a new node for tunables documentation and add notes for the > malloc tunables. > > * manual/tunables.texi: New chapter. > * manual/Makefile (chapters): Add it. > * manual/probes.texi (@node): Point to the Tunables chapter. > --- > manual/Makefile | 3 +- > manual/probes.texi | 2 +- > manual/tunables.texi | 185 +++++++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 188 insertions(+), 2 deletions(-) > create mode 100644 manual/tunables.texi > > diff --git a/manual/Makefile b/manual/Makefile > index f2f694f..ecc2bf6 100644 > --- a/manual/Makefile > +++ b/manual/Makefile > @@ -38,7 +38,8 @@ chapters = $(addsuffix .texi, \ > message search pattern io stdio llio filesys \ > pipe socket terminal syslog math arith time \ > resource setjmp signal startup process ipc job \ > - nss users sysinfo conf crypt debug threads probes) > + nss users sysinfo conf crypt debug threads \ > + probes tunables) > add-chapters = $(wildcard $(foreach d, $(add-ons), ../$d/$d.texi)) > appendices = lang.texi header.texi install.texi maint.texi platform.texi \ > contrib.texi > diff --git a/manual/probes.texi b/manual/probes.texi > index 237a918..eb91c62 100644 > --- a/manual/probes.texi > +++ b/manual/probes.texi > @@ -1,5 +1,5 @@ > @node Internal Probes > -@c @node Internal Probes, , POSIX Threads, Top > +@c @node Internal Probes, Tunables, POSIX Threads, Top > @c %MENU% Probes to monitor libc internal behavior > @chapter Internal probes > > diff --git a/manual/tunables.texi b/manual/tunables.texi > new file mode 100644 > index 0000000..1f1f9fc > --- /dev/null > +++ b/manual/tunables.texi > @@ -0,0 +1,185 @@ > +@node Tunables > +@c @node Tunables, , Internal Probes, Top > +@c %MENU% Tunable switches to alter libc internal behavior > +@chapter Tunables Maybe some Concept Index entries here? At least: @cindex tunables > + > +Tunables is a feature in @theglibc{} that allows application authors and @dfn{Tunables} are > +distribution maintainers to alter the runtime library behaviour to match > +their workload. These are implemented as a set of switches that may be > +modified in different ways. The current default method to do this is via > +the @env{GLIBC_TUNABLES} environment variable by setting it to a string > +of colon-separated @var{name}=@var{value} pairs. For example, the following > +environment variable export enables malloc checking and sets the malloc > +trim threshold to 128 bytes. Anything with "the following", I usually end with a ":". > + > +@code{export GLIBC_TUNABLES=glibc.malloc.trim_threshold=128:glibc.malloc.check=3} Could be a @smallexample, and omit the @code{}. > + > +Tunables are not part of the @glibcadj{} stable ABI, and they are > +subject to change or removal across releases. Additionally, the method to > +modify tunable values may change between releases and across distributions. > +It is possible to implement multiple `frontends' for the tunables allowing > +distributions to choose their preferred method at build time. > + > +Finally, the set of tunables available may vary between distributions as > +the tunables feature allows distributions to add their own tunables under > +their own namespace. > + > +@menu > +* Tunable names:: The structure of a tunable name > +* Memory Allocation Tunables:: Tunables in the memory allocation subsystem > +@end menu > + > +@node Tunable names > +@section Tunable names More proposed @cindex entries: @cindex tunable names @cindex tunable namespaces > + > +A tunable name is split into three components, a top namespace, a tunable > +namespace and the tunable name. The top namespace for tunables > +implemented in @theglibc{} project is @code{glibc}. Distributions that -project > +choose to add custom tunables in their maintained versions of @theglibc{} > +may choose to do so under their own top namespace. > + > +The tunable namespace is a logical grouping of tunables in a single > +module. This currently holds no special significance, although that may > +change in future. the future > + > +The tunable name is the actual name of the tunable. It is possible that > +different tunable namespaces may have tunables within them that have the > +same name, likewise for top namespaces. Hence, we only support > +identification of tunables by their full name, i.e. with the top > +namespace, tunable namespace and tunable name, separated by periods. > + > +@node Memory Allocation Tunables > +@section Memory Allocation Tunables Some @cindex entries here would be good, particularly tying together malloc + tunables; e.g.: @cindex memory allocation tunables @cindex malloc, tunables @cindex tunables, malloc > + > +@deftp Namespace glibc.malloc > +Memory allocation behaviour can be modified by setting any of the > +following tunables in the @code{malloc} namespace. > +@end deftp > + > +@deftp Tunable glibc.malloc.check > +This tunable supersedes the @code{MALLOC_CHECK_} environment variable and is > +identical in features. > + > +Setting this tunable enables a special (less efficient) memory allocator for > +the malloc family of functions that is designed to be tolerant against simple > +errors such as double calls of free with the same argument, or overruns of a > +single byte (off-by-one bugs). Not all such errors can be protected against, > +however, and memory leaks can result. The following list describes the values > +that this tunable can take and the effect they have on malloc functionality. > + > +@itemize @bullet > +@item @code{0} Disable all error reporting. The alternate allocator is > +selected and heap corruption detection is in place, but any such errors > +detected are ignored. This is currently a supported use, but is not > +recommended. > +@item @code{1} Report errors. The alternate allocator is selected and heap > +corruption, if detected, is reported as diagnostic messages to @code{stderr} > +and the program continues execution. > +@item @code{2} Abort on errors. The alternate allocator is selected and if > +heap corruption is detected, the program is ended immediately by calling > +@code{abort}. > +@item @code{3} Fully enabled. The alternate allocator is selected and is fully > +functional. That is, if heap corruption is detected, a verbose diagnostic > +message is printed to @code{stderr} and the program is ended by calling > +@code{abort}. > +@end itemize > + > +Like @env{MALLOC_CHECK_}, @code{glibc.malloc.check} has a problem in that it > +diverges from normal program behavior by writing to @code{stderr}, which could > +by exploited in SUID and SGID binaries. Therefore, @code{glibc.malloc.check} > +is disabled by default for SUID and SGID binaries. This can be enabled again > +by the system administrator by adding a file @file{/etc/suid-debug}; the > +content of the file could be anything or even empty. > +@end deftp > + > +@deftp Tunable glibc.malloc.top_pad > +This tunable supersedes the @code{MALLOC_TOP_PAD_} environment variable and is > +identical in features. > + > +This tunable determines the amount of extra memory to obtain from the system "amount of extra memory in bytes..."; I think it would be nice if the first sentence mentioned the units. > +when any of the arenas need to be extended. It also specifies the number of > +bytes to retain when shrinking any of the arenas. This provides the necessary > +hysteresis in heap size such that excessive amounts of system calls can be > +avoided. > + > +The default value of this tunable is @samp{0}. > +@end deftp > + > +@deftp Tunable glibc.malloc.perturb > +This tunable supersedes the @code{MALLOC_PERTURB_} environment variable and is > +identical in features. > + > +If set to a non-zero value, memory blocks are initialized with values depending > +on some low order bits of this tunable when they are allocated (except when > +allocated by calloc) and freed. This can be used to debug the use of > +uninitialized or freed heap memory. Note that this option does not guarantee > +that the freed block will have any specific values. It only guarantees that the > +content the block had before it was freed will be overwritten. This seems non-deterministic and leaves me wondering which low-order bits, and how to select my values so I know what gets written. > + > +The default value of this tunable is @samp{0}. > +@end deftp > + > +@deftp Tunable glibc.malloc.mmap_threshold > +This tunable supersedes the @code{MALLOC_MMAP_THRESHOLD_} environment variable > +and is identical in features. > + > +When this tunable is set, all chunks larger than this value are allocated > +outside the normal heap, using the @code{mmap} system call. This way it is > +guaranteed that the memory for these chunks can be returned to the system on > +@code{free}. Note that requests smaller than this threshold might still be > +allocated via @code{mmap}. Units here, too. > + > +If this tunable is not set, the default value is set as 128 KB and the > +threshold is adjusted dynamically to suit the allocation patterns of the > +program. If the tunable is set, the dynamic adjustment is disabled and the > +value is set as static. > +@end deftp > + > +@deftp Tunable glibc.malloc.trim_threshold > +This tunable supersedes the @code{MALLOC_TRIM_THRESHOLD_} environment variable > +and is identical in features. > + > +The value of this tunable is the minimum size (in bytes) of the top-most, > +releasable chunk in an arena that will trigger a system call in order to return > +memory to the system from that arena. > + > +If this tunable is not set, the default value is set as 128 KB and the set to I thought this was in bytes! ;) > +threshold is adjusted dynamically to suit the allocation patterns of the > +program. If the tunable is set, the dynamic adjustment is disabled and the > +value is set as static. > +@end deftp > + > +@deftp Tunable glibc.malloc.mmap_max > +This tunable supersedes the @code{MALLOC_MMAP_MAX_} environment variable and is > +identical in features. > + > +The value of this tunable is maximum number of chunks to allocate with > +@code{mmap}. Setting this to zero disables all use of @code{mmap}. > + > +The default value of this tunable is @samp{65536}. > +@end deftp > + > +@deftp Tunable glibc.malloc.arena_test > +This tunable supersedes the @code{MALLOC_ARENA_TEST} environment variable and is > +identical in features. > + > +The @code{glibc.malloc.arena_test} tunable specifies the number of arenas that > +can be created before the test on the limit to the number of arenas is > +conducted. The value is ignored if @code{glibc.malloc.arena_max} is set. > + > +The default value of this tunable is 2 for 32-bit systems and 8 for 64-bit > +systems. > +@end deftp > + > +@deftp Tunable glibc.malloc.arena_max > +This tunable supersedes the @code{MALLOC_ARENA_MAX} environment variable and is > +identical in features. > + > +This tunable sets the number of arenas to use in a process regardless of the > +number of cores in the system. > + > +The default value of this tunable is @code{0}, meaning that the limit on the > +number of arenas is determined by the number of CPU cores online. For 32-bit > +systems the limit is twice the number of cores online and on 64-bit systems, it > +is 8 times the number of cores online. > +@end deftp Whitespace issues throughout; noticed a lot of single-space separated sentences. Not sure that it matters in the rendered output, but it's nice for source readability. I like how you made this its own chapter. I think it's a good start. Rical