From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-oi1-x22d.google.com (mail-oi1-x22d.google.com [IPv6:2607:f8b0:4864:20::22d]) by sourceware.org (Postfix) with ESMTPS id 497453858C54 for ; Mon, 27 Mar 2023 20:26:29 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 497453858C54 Authentication-Results: sourceware.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=linaro.org Received: by mail-oi1-x22d.google.com with SMTP id bk5so7358014oib.6 for ; Mon, 27 Mar 2023 13:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1679948788; h=content-transfer-encoding:in-reply-to:organization:from:references :to:content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=hCEncde8cIMfUhaK/3MBeYPM802JLLFg4q9cRZKvFbo=; b=cUxbVhjK9zxL6pRxQSxV3Hdg64dC1KqnVx1akhwlsTY6lj7I18Fskbtq6TkkNEBWoY BzWbEK0F1Nn5x4v7DSjsdKuyUa7rCDQgW/KcelCRwR6HsUGgLDX7KS3Iv4FhykFUIBWF wXAdb0Be6fYA+YomuoI6mgF7/RAn1n+vq6Bph1VaiBQzF4HH+EFhmgE7ZfBkKOMsnZhc UPcxycXyGi7a6Tuez7ifcfONFd8JGD1pGyJEpIujcthqCaBdghM3/Mt7+G6q9CLMRNmb eyw+gT4CrEKJ3P3SqoSuME6mkbas5UHHKT3hZpUqFAlBe6XbZTjL96GQHaXDDLEhGvQx 7T8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679948788; h=content-transfer-encoding:in-reply-to:organization:from:references :to:content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hCEncde8cIMfUhaK/3MBeYPM802JLLFg4q9cRZKvFbo=; b=2r0mflBpLdgy+y+9PROKujxu7S7fKwSLBharli46Rlwe9DDbPUEfpQeJoPXk5iEOyJ seUCYtzYMs29e71LkkE2pFMEeM/U7jF/jf834EcuG4nud5TVYSpSuB1UzDhCIETqxnIK NKs0O5Glz9Pu46CE9dzgY25IWlnxBb85JUNqNxJT/diWtAxd0fJyF61uj7argAPhzYSF AwtMODY+aPq5GPC0Q/ifmvtlnSjZ/Ab/SrKq3UzHaPwe2nynSJ2+FKRHx8X1hTwuYy4l bI7DLssOmaW+GUKa2UnOZnWcJIFNmFyGso17HRNijpXgqIwCVlmCm43hbqOaXQiwqucK cLyg== X-Gm-Message-State: AO0yUKXEmQtixGBlDmRQOo859cg+TuMuv9G0d6F1RxtIAZ5t++W65I7f 6gbKY6lqVqOLbjsypEt8vG0U79PLF6CpIKjqFoP3vg== X-Google-Smtp-Source: AK7set9QbVFuMUmVYGOYW93HCRDbEKhxwDAyVDZ2CEUNepz5ZKZo7TeaJA5pbCpt7QzdDZ1D8gpHnQ== X-Received: by 2002:a05:6808:15a7:b0:386:e073:6996 with SMTP id t39-20020a05680815a700b00386e0736996mr7767509oiw.26.1679948788094; Mon, 27 Mar 2023 13:26:28 -0700 (PDT) Received: from ?IPV6:2804:1b3:a7c1:60f9:9d7f:7b90:a6ae:401d? ([2804:1b3:a7c1:60f9:9d7f:7b90:a6ae:401d]) by smtp.gmail.com with ESMTPSA id v41-20020a4a8c6c000000b0052532296449sm11681801ooj.20.2023.03.27.13.26.26 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 27 Mar 2023 13:26:27 -0700 (PDT) Message-ID: <8d44ed82-72c3-9ff4-47d1-8822ec9225c9@linaro.org> Date: Mon, 27 Mar 2023 17:26:25 -0300 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [PATCH v3 1/1] Created tunable to force small pages on stack allocation. Content-Language: en-US To: libc-alpha@sourceware.org References: <20230321145116.571178-1-cupertino.miranda@oracle.com> <20230321145116.571178-2-cupertino.miranda@oracle.com> From: Adhemerval Zanella Netto Organization: Linaro In-Reply-To: <20230321145116.571178-2-cupertino.miranda@oracle.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-12.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,GIT_PATCH_0,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: On 21/03/23 11:51, Cupertino Miranda via Libc-alpha wrote: > Created tunable glibc.pthread.stack_hugetlb to control when hugepages > can be used for stack allocation. > In case THP are enabled and glibc.pthread.stack_hugetlb is set to > 0, glibc will madvise the kernel not to use allow hugepages for stack > allocations. > > Changed from v1: > - removed the __malloc_thp_mode calls to check if hugetlb is > enabled. > > Changed from v2: > - Added entry in manual/tunables.texi > - Fixed tunable default to description > - Code style corrections. > --- > manual/tunables.texi | 11 +++++++++++ > nptl/allocatestack.c | 6 ++++++ > nptl/nptl-stack.c | 1 + > nptl/nptl-stack.h | 3 +++ > nptl/pthread_mutex_conf.c | 8 ++++++++ > sysdeps/nptl/dl-tunables.list | 6 ++++++ > 6 files changed, 35 insertions(+) > > diff --git a/manual/tunables.texi b/manual/tunables.texi > index 70dd2264c5..de6fd9d6b3 100644 > --- a/manual/tunables.texi > +++ b/manual/tunables.texi > @@ -459,6 +459,17 @@ registration on behalf of the application. > Restartable sequences are a Linux-specific extension. > @end deftp > > +@deftp Tunable glibc.pthread.stack_hugetlb > +This tunable allows to configure stack allocation never to use hugetlbs. I think we should improve the wording here to make it clear this for the stack allocated for newly created threads even though the tunables name implies it ('stack allocation' might infer that all stack allocation, even for single-thread programs) and that does not affect stack set by pthread_attr_setstack. > +This is more of an RSS optimization, for example in scenarios where many > +threads get created, keeping RSS to a minimum, but also allowing hugestlbs to > +be used for malloc allocations. > + > +The default is @samp{1} and preservs glibc default behaviour. > +When set to @samp{0}, it advises the kernel not to use hugetlbs for stack > +allocation. > +@end deftp > + > @node Hardware Capability Tunables > @section Hardware Capability Tunables > @cindex hardware capability tunables > diff --git a/nptl/allocatestack.c b/nptl/allocatestack.c > index c7adbccd6f..f9d8cdfd08 100644 > --- a/nptl/allocatestack.c > +++ b/nptl/allocatestack.c > @@ -369,6 +369,12 @@ allocate_stack (const struct pthread_attr *attr, struct pthread **pdp, > if (__glibc_unlikely (mem == MAP_FAILED)) > return errno; > > + /* Do madvise in case the tunable glibc.pthread.stack_hugetlb is > + set to 0, disabling hugetlb. */ > + if (__glibc_unlikely (__nptl_stack_hugetlb == 0) > + && __madvise (mem, size, MADV_NOHUGEPAGE) != 0) > + return errno; > + > /* SIZE is guaranteed to be greater than zero. > So we can never get a null pointer back from mmap. */ > assert (mem != NULL); > diff --git a/nptl/nptl-stack.c b/nptl/nptl-stack.c > index 5eb7773575..e829711cb5 100644 > --- a/nptl/nptl-stack.c > +++ b/nptl/nptl-stack.c > @@ -21,6 +21,7 @@ > #include > > size_t __nptl_stack_cache_maxsize = 40 * 1024 * 1024; > +int32_t __nptl_stack_hugetlb = 1; > > void > __nptl_stack_list_del (list_t *elem) > diff --git a/nptl/nptl-stack.h b/nptl/nptl-stack.h > index 34f8bbb15e..cf90b27c2b 100644 > --- a/nptl/nptl-stack.h > +++ b/nptl/nptl-stack.h > @@ -27,6 +27,9 @@ > /* Maximum size of the cache, in bytes. 40 MiB by default. */ > extern size_t __nptl_stack_cache_maxsize attribute_hidden; > > +/* Should allow stacks to use hugetlb. (1) is default. */ > +extern int32_t __nptl_stack_hugetlb; > + > /* Check whether the stack is still used or not. */ > static inline bool > __nptl_stack_in_use (struct pthread *pd) > diff --git a/nptl/pthread_mutex_conf.c b/nptl/pthread_mutex_conf.c > index 329c4cbb8f..60ef9095aa 100644 > --- a/nptl/pthread_mutex_conf.c > +++ b/nptl/pthread_mutex_conf.c > @@ -45,6 +45,12 @@ TUNABLE_CALLBACK (set_stack_cache_size) (tunable_val_t *valp) > __nptl_stack_cache_maxsize = valp->numval; > } > > +static void > +TUNABLE_CALLBACK (set_stack_hugetlb) (tunable_val_t *valp) > +{ > + __nptl_stack_hugetlb = (int32_t) valp->numval; > +} > + > void > __pthread_tunables_init (void) > { > @@ -52,5 +58,7 @@ __pthread_tunables_init (void) > TUNABLE_CALLBACK (set_mutex_spin_count)); > TUNABLE_GET (stack_cache_size, size_t, > TUNABLE_CALLBACK (set_stack_cache_size)); > + TUNABLE_GET (stack_hugetlb, int32_t, > + TUNABLE_CALLBACK (set_stack_hugetlb)); > } > #endif > diff --git a/sysdeps/nptl/dl-tunables.list b/sysdeps/nptl/dl-tunables.list > index bd1ddb121d..4cde9500b6 100644 > --- a/sysdeps/nptl/dl-tunables.list > +++ b/sysdeps/nptl/dl-tunables.list > @@ -33,5 +33,11 @@ glibc { > maxval: 1 > default: 1 > } > + stack_hugetlb { > + type: INT_32 > + minval: 0 > + maxval: 1 > + default: 1 > + } > } > }