From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oa1-x31.google.com (mail-oa1-x31.google.com [IPv6:2001:4860:4864:20::31]) by sourceware.org (Postfix) with ESMTPS id 94F753856DC8 for ; Thu, 20 Apr 2023 16:55:53 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 94F753856DC8 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org Received: by mail-oa1-x31.google.com with SMTP id 586e51a60fabf-187d9c205e9so711218fac.3 for ; Thu, 20 Apr 2023 09:55:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1682009752; x=1684601752; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:from:to:cc:subject:date:message-id:reply-to; bh=+ZNkSPHEJ9TiXgcoQtc6KevHK3GDrkjPtw0jOZixKtU=; b=pRY7opfMKAJgRPKTrpfOBiv45/5YF5Em5R2wtxdADH+3j4TokXWZjBX/3GprReWqI9 okaqSpszKqcCL/6DI+mXuQDnMYLKjalZPm3+kyMY/oFZgM8WhOUsZCjCvwWRA0gjMUQC 2zRXCNehTQP71mIqljG7Z/GjBl4+fGF3xIqOePfFVaAxyJcqcdywprICnBcU5RIDEvh4 kSye/elknZ+ff4Att3YEVl8A6N3+2liTYPCemDZbUQGXXntQo1ihmkrk/FOHmNjlvo6J 2S8QiY/tioIdzGRzNHZmAsEspvPyFhS0ziE/AkEnorW7r7UFVAOEA9FbzomC4S2uw0ql wslg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682009752; x=1684601752; h=content-transfer-encoding:in-reply-to:organization:from:references :cc:to:content-language:subject:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+ZNkSPHEJ9TiXgcoQtc6KevHK3GDrkjPtw0jOZixKtU=; b=FcpSLGY0JPV1xPT0A8MQDnJuvwZEclj5js+vilosWeERo9Szlk9IGpzyHwEfdGJgHV YuDj8PMe6wAGAWiMTKPoeA9ySJlNqOmxicFOOcSTbiHFukon6XxT9rzy9TkstPXiCiAg R5eIt3d3DZMX1D3LkrYRJWcXoe3RLCkC+7BBl81DxMAFahZuJxTITzAafOXqRVxwCE0K 7IaNfMkrXEcfcCCFSNc9FnXQWm0Ockxh4feXVgcIg2s5azK+9HkV5a/8yOBhO0j1V3cL SN29owoiidxYn3lLhXey7tIznH8dQawVWw8SG9WOn12kkTaMw6TSPDkT5hfMII3aqrKr TDSQ== X-Gm-Message-State: AAQBX9d+fNCTL7pih0k12yBMHGe94q0peCYr4TPVLvqjGr2WJjZztllS QmDDcnjMZWQaP8UibDPvqcXstQ== X-Google-Smtp-Source: AKy350YDhYhvG/q1z5HKhRDM//eYpej53HsOuSZ3BwXIT//xj1Cb2m1hyr3g9V+X+ErzAymoIsPw3A== X-Received: by 2002:a05:6870:80c9:b0:187:9cf0:cc34 with SMTP id r9-20020a05687080c900b001879cf0cc34mr1564242oab.49.1682009752536; Thu, 20 Apr 2023 09:55:52 -0700 (PDT) Received: from ?IPV6:2804:1b3:a7c3:333:20b7:b016:1b7f:fd25? ([2804:1b3:a7c3:333:20b7:b016:1b7f:fd25]) by smtp.gmail.com with ESMTPSA id b1-20020a056870b24100b0018045663fc5sm899378oam.48.2023.04.20.09.55.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 20 Apr 2023 09:55:51 -0700 (PDT) Message-ID: <257edd00-44f5-0da9-39dc-2ef21341a036@linaro.org> Date: Thu, 20 Apr 2023 13:55:49 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: [PATCH v6 1/1] Created tunable to force small pages on stack allocation. Content-Language: en-US To: Cupertino Miranda , libc-alpha@sourceware.org Cc: jose.marchesi@oracle.com, elena.zannoni@oracle.com References: <20230414151220.52040-1-cupertino.miranda@oracle.com> <20230414151220.52040-2-cupertino.miranda@oracle.com> From: Adhemerval Zanella Netto Organization: Linaro In-Reply-To: <20230414151220.52040-2-cupertino.miranda@oracle.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-13.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 14/04/23 12:12, Cupertino Miranda via Libc-alpha wrote: > Created tunable glibc.pthread.stack_hugetlb to control when hugepages > can be used for stack allocation. > In case THP are enabled and glibc.pthread.stack_hugetlb is set to > 0, glibc will madvise the kernel not to use allow hugepages for stack > allocations. > > Changed from v1: > - removed the __malloc_thp_mode calls to check if hugetlb is > enabled. > > Changed from v2: > - Added entry in manual/tunables.texi > - Fixed tunable default to description > - Code style corrections. > > Changes from v3: > - Improve tunables.texi. > > Changes from v4: > - Improved text in tunables.texi by suggestion of Adhemerval. > > Changes from v5: > - Added a new entry in NEWS. LGTM, thanks. Reviewed-by: Adhemerval Zanella > --- > NEWS | 3 +++ > manual/tunables.texi | 15 +++++++++++++++ > nptl/allocatestack.c | 6 ++++++ > nptl/nptl-stack.c | 1 + > nptl/nptl-stack.h | 3 +++ > nptl/pthread_mutex_conf.c | 8 ++++++++ > sysdeps/nptl/dl-tunables.list | 6 ++++++ > 7 files changed, 42 insertions(+) > > diff --git a/NEWS b/NEWS > index 83d082afad..40964d2ee0 100644 > --- a/NEWS > +++ b/NEWS > @@ -21,6 +21,9 @@ Major new features: > > * PRIb* and PRIB* macros from C2X have been added to . > > +* A new tunable, glibc.pthread.stack_hugetlb, can be used to disable > + Transparent Huge Pages (THP) in stack allocation at pthread_create. > + > Deprecated and removed features, and other changes affecting compatibility: > > * In the Linux kernel for the hppa/parisc architecture some of the > diff --git a/manual/tunables.texi b/manual/tunables.texi > index 70dd2264c5..130f94b2bc 100644 > --- a/manual/tunables.texi > +++ b/manual/tunables.texi > @@ -459,6 +459,21 @@ registration on behalf of the application. > Restartable sequences are a Linux-specific extension. > @end deftp > > +@deftp Tunable glibc.pthread.stack_hugetlb > +This tunable controls whether to use Huge Pages in the stacks created by > +@code{pthread_create}. This tunable only affects the stacks created by > +@theglibc{}, it has no effect on stack assigned with > +@code{pthread_attr_setstack}. > + > +The default is @samp{1} where the system default value is used. Setting > +its value to @code{0} enables the use of @code{madvise} with > +@code{MADV_NOHUGEPAGE} after stack creation with @code{mmap}. > + > +This is a memory utilization optimization, since internal glibc setup of either > +the thread descriptor and the guard page might force the kernel to move the > +thread stack originally backup by Huge Pages to default pages. > +@end deftp > + > @node Hardware Capability Tunables > @section Hardware Capability Tunables > @cindex hardware capability tunables > diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c > index c7adbccd6f..f9d8cdfd08 100644 > --- a/nptl/allocatestack.c > +++ b/nptl/allocatestack.c > @@ -369,6 +369,12 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, > if (__glibc_unlikely (mem == MAP_FAILED)) > return errno; > > + /* Do madvise in case the tunable glibc.pthread.stack_hugetlb is > + set to 0, disabling hugetlb. */ > + if (__glibc_unlikely (__nptl_stack_hugetlb == 0) > + && __madvise (mem, size, MADV_NOHUGEPAGE) != 0) > + return errno; > + > /* SIZE is guaranteed to be greater than zero. > So we can never get a null pointer back from mmap. */ > assert (mem != NULL); > diff --git a/nptl/nptl-stack.c b/nptl/nptl-stack.c > index 5eb7773575..e829711cb5 100644 > --- a/nptl/nptl-stack.c > +++ b/nptl/nptl-stack.c > @@ -21,6 +21,7 @@ > #include > > size_t __nptl_stack_cache_maxsize = 40 * 1024 * 1024; > +int32_t __nptl_stack_hugetlb = 1; > > void > __nptl_stack_list_del (list_t *elem) > diff --git a/nptl/nptl-stack.h b/nptl/nptl-stack.h > index 34f8bbb15e..cf90b27c2b 100644 > --- a/nptl/nptl-stack.h > +++ b/nptl/nptl-stack.h > @@ -27,6 +27,9 @@ > /* Maximum size of the cache, in bytes. 40 MiB by default. */ > extern size_t __nptl_stack_cache_maxsize attribute_hidden; > > +/* Should allow stacks to use hugetlb. (1) is default. */ > +extern int32_t __nptl_stack_hugetlb; > + > /* Check whether the stack is still used or not. */ > static inline bool > __nptl_stack_in_use (struct pthread *pd) > diff --git a/nptl/pthread_mutex_conf.c b/nptl/pthread_mutex_conf.c > index 9133384d47..6517899718 100644 > --- a/nptl/pthread_mutex_conf.c > +++ b/nptl/pthread_mutex_conf.c > @@ -44,6 +44,12 @@ TUNABLE_CALLBACK (set_stack_cache_size) (tunable_val_t *valp) > __nptl_stack_cache_maxsize = valp->numval; > } > > +static void > +TUNABLE_CALLBACK (set_stack_hugetlb) (tunable_val_t *valp) > +{ > + __nptl_stack_hugetlb = (int32_t) valp->numval; > +} > + > void > __pthread_tunables_init (void) > { > @@ -51,4 +57,6 @@ __pthread_tunables_init (void) > TUNABLE_CALLBACK (set_mutex_spin_count)); > TUNABLE_GET (stack_cache_size, size_t, > TUNABLE_CALLBACK (set_stack_cache_size)); > + TUNABLE_GET (stack_hugetlb, int32_t, > + TUNABLE_CALLBACK (set_stack_hugetlb)); > } > diff --git a/sysdeps/nptl/dl-tunables.list b/sysdeps/nptl/dl-tunables.list > index bd1ddb121d..4cde9500b6 100644 > --- a/sysdeps/nptl/dl-tunables.list > +++ b/sysdeps/nptl/dl-tunables.list > @@ -33,5 +33,11 @@ glibc { > maxval: 1 > default: 1 > } > + stack_hugetlb { > + type: INT_32 > + minval: 0 > + maxval: 1 > + default: 1 > + } > } > }