From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from www458.your-server.de (www458.your-server.de [136.243.165.62]) by sourceware.org (Postfix) with ESMTPS id EC0203857C5A; Wed, 23 Sep 2020 22:13:37 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org EC0203857C5A Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=conp-solutions.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nmanthey@conp-solutions.com Received: from sslproxy01.your-server.de ([78.46.139.224]) by www458.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92.3) (envelope-from ) id 1kLD15-0005Pi-Ox; Thu, 24 Sep 2020 00:13:35 +0200 Received: from [92.117.121.181] (helo=nmanthey-ThinkPad-X1-Carbon-3rd.fritz.box) by sslproxy01.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kLD15-000TLx-8F; Thu, 24 Sep 2020 00:13:35 +0200 From: Norbert Manthey To: libc-alpha@sourceware.org Cc: Siddhesh Poyarekar , DJ Delorie , Carlos O'Donell , Norbert Manthey Subject: [[PATCH] v2 0/1] malloc/realloc with transparent huge page support Date: Thu, 24 Sep 2020 00:12:59 +0200 Message-Id: <20200923221300.162450-1-nmanthey@conp-solutions.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <877dxsw1eu.fsf@mid.deneb.enyo.de> References: <877dxsw1eu.fsf@mid.deneb.enyo.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Authenticated-Sender: nmanthey@conp-solutions.com X-Virus-Scanned: Clear (ClamAV 0.102.4/25936/Wed Sep 23 15:55:51 2020) X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Sep 2020 22:13:39 -0000 Dear all, rev2: * rebased on top of origin/master * use glibc tunables instead of environment variables * make size of huge page configurable during compile time * copyright disclaimers have been signed I have been looking for a way to enable huge pages in a user land process without privileges, and without modifying the source of the process. Huge pages had to be used explicitly via linking against special libraries. An alternative are transparent huge pages, which are easier to obtain, but there are no guarantees of success. Using transparent huge pages can also result in slowdowns. Many Linux based distributions enable transparent huge pages via madvise, so that user land processes. With the madvise system call, a process can control whether allocated memory should be mapped with huge pages. However, the process typically only sees the allocations returned via a single malloc or realloc invocation, most of them being rather small. For a larger impact, I modified the way allocations are done behind the scenes. For the allocated memory, I changed the properties to (1) align to 2M instead of 4K, and (2) to call the madvise system call after the brk system call, to signal that the used memory area should be mapped with huge pages. The default behavior is not modified. The modification can currently be enabled, via glibc tunables, or the related environment variables ### Performance Testing Done This change has been tested on top of glibc 2.23 in an Ubuntu 16.04 environment for programs that benefit from huge pages due to their large memory usage and pseudo-random memory access patterns (e.g. SAT solvers, model checkers, optimization tools an others). More details on the performance improvements for these tools can be found in https://arxiv.org/abs/2004.14378, e.g. page 9. I would look forward to recommendataion on scenarios to test. #### glibc tests Both "make check" and "make xcheck" have not introduced regressions. I tested on a Ubuntu 20.04, with gcc-9.3 and an Intel(R) Core(TM) i7-5600U CPU. ### Reproduction and testing How to use the changes via environment variables: (1) build glibc, e.g. into directory 'build' (2) GLIBC=$(readlink -e build) Get a copy of the one-line-scan tool: https://github.com/awslabs/one-line-scan Go to the project to be linked against the local library, and run the following: ONELINESCAN= "$ONELINESCAN"/one-line-scan --no-gotocc \ --plain --extra-cflag-set \ "-L${GLIBC} -g -Wl,-rpath=${GLIBC}:${GLIBC}/math:${GLIBC}/elf:${GLIBC}/dlfcn:${GLIBC}/nss:${GLIBC}/nis:${GLIBC}/rt:${GLIBC}/resolv:${GLIBC}/crypt:${GLIBC}/nptl:${GLIBC}/dfp -Wl,--dynamic-linker=${GLIBC}/elf/ld.so -Wl,${GLIBC}/libc_nonshared.a" \ -- make r -j 6 As an alternative, the modified glibc could be installed in a docker container or virtual machine to either run an dynamically linked application, or statically link an application to run the changes. For Ubuntu 18.04, a docker environment is provided for example in the repository of MergeSat: https://github.com/conp-solutions/mergesat/blob/master/Dockerfile ### GPL License The copyright disclaimers have been signed as required for glibc contribution. Best, Norbert Norbert Manthey (1): malloc: support transparent huge pages elf/dl-tunables.list | 12 ++++ malloc/arena.c | 19 +++--- malloc/malloc.c | 159 ++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 170 insertions(+), 20 deletions(-) -- 2.25.1