From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from www458.your-server.de (www458.your-server.de [136.243.165.62]) by sourceware.org (Postfix) with ESMTPS id 59CCA385DC32; Mon, 4 May 2020 06:58:10 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 59CCA385DC32 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=conp-solutions.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=nmanthey@conp-solutions.com Received: from sslproxy02.your-server.de ([78.47.166.47]) by www458.your-server.de with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89_1) (envelope-from ) id 1jVV3J-0004S6-6T; Mon, 04 May 2020 08:58:09 +0200 Received: from [92.117.118.30] (helo=[192.168.178.34]) by sslproxy02.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jVV3I-0000gN-Lv; Mon, 04 May 2020 08:58:08 +0200 Subject: Re: [[RFC][PATCH] v1 0/2] malloc/realloc with transparent huge page support To: Florian Weimer Cc: libc-alpha@sourceware.org, Siddhesh Poyarekar References: <20200503220708.2048246-1-nmanthey@conp-solutions.com> <87ftcgz58n.fsf@mid.deneb.enyo.de> From: Norbert Manthey Autocrypt: addr=nmanthey@conp-solutions.com; prefer-encrypt=mutual; keydata= mQINBFoR7HQBEAC8bqNVzZZOYnUhGx1NvvhM0Xd4g0etXQCRaJ6v+bySUpbwDqNCMt/bxAbW bpOdHX9LlAFzLPORs/Hqa3GiOdIRRUmmYbl4SjceTqZz1Oh4/mY1z4WeLG0az6+W4B3/B2EX uDaY2WX0FtS/GlGZll5KbWRF5P1BaVeKSccy20dC0AC3wOm0vwKLixZrK4ukPpTdphg6Klnx a3yRNqBSbcXAh6Xuz3qs395mgZYxNGAr8tmybAJGWWCNKx4ePQczEQ1oUaELJtNiBrLr9L2k S8XOr8jn3DD2gs1ph+aT5ThzURyK8s1S7U+czthWyydSASjqC1WINsdhjnCh2He1qtHPtvPx kj5lHMRlXmbhMBYewwlZgD++3ZlgFeJ6OfxoWWXKZhceO9+BrbCbyXYjFadIHeDUEV+1Gcle Z+5W1exRXPrbZ1w3Tv4HLaz544o2EGvO3ltSpo+hiz7NhE+2fjgvQB9VQm2LUp9yaVeKmHdi u9Sq6y7eO6nFk2SQoGo86rCo9PI2TwBgG65zJYuD6h6GfiDV2G0O1J7reeQLriDI5O7OXGTb DrZyARWfAvi+UMrQKcbbmcwEX9ZHfZod3BEfDPWhfK4z9Em3dBFyMdoMvhUBlLvrovCixiQ+ /uN7nOepz4agKYNLct+RnQA1uoBa+qDpOnuKMVcIgfJvLTiWFQARAQABtDFEci4gTm9yYmVy dCBNYW50aGV5IDxubWFudGhleUBjb25wLXNvbHV0aW9ucy5jb20+iQI9BBMBCAAnBQJaEex0 AhsjBQkJZgGABQsJCAcCBhUICQoLAgQWAgMBAh4BAheAAAoJEKBSnOLnk4Rrq2gP/2sW27Nh 9hOpCI1n2k+3c/p6Ljq9CEh1xfOaz380asLE4+48pT479CXe0M1FxLay3ap/Zootkbhl3B31 Td37a4OViwtojcby9sjNndaHYfWQBSbh7hRYSsAms+ddOBhIBWxnYqXVvmgigARZGZdIW8sI WYK+FHlQufu9yb3eRwrwHb04S7UBrH2n63YjcMFcxGxOeJE5SzcPtuuiVqBnv9P8SWjUwvVz sXIsggi/nl2d9u2lGAoXekLmALGylxI+LQsgxaal+PHI5KJ2Rb0CNdcRpJyBhPWOpM6V9BnM j6gjY37s2yYIWYfnx+tQSpprlh9GzLHe8BMyHCZTqoCbJteN3yeQ2TvW0/cbe8Gza1C/T40n 72xAqbsX54SG3mroJO7Kg2w1aJug4eujaDkg1rOj/l3mul99VKiHfzDP6PNNpnqlLmRlo+CA 09snmhrCwhXBX8PqVrDZT8zr8SM/yVci7aIgoviXwaprPZ3utwxh06azp+80jsyJDbhlpI4E eqoT99hH5875ru3NCghOeebOfgjvLZ27rU8EUbyBjmbNcdB8Ri6RjWh90+PPdp2nxlAhwfe4 j19n1vKF1wgnbwozqCfbvjx2akobJr8QcexhN+bGpL8NSM8h+pBVcmAmVF+yG26JoUNNkq72 AfeihtumG9ouRmhuJNplTTVTxareuQINBFoR7HQBEACy3Zv/hcDc6pQYhjYG6I7qU1eagnhh OOIWqtfMb1TJbgG018iJdlEd8lMpMonID3DLRpTUBmCh64rRglebh4wYlYo5iH+QHHWIwkHg TnLWI+Q2yLr7jJKvPVHzrCOtMyBxc4VIQkRN+msspqLx3kf3AYhOJ2zfmwoW72UEpCQqAEvf UNUXNgSpj+n1h/96z2CZuD8wyR3TrOV4N+074eTM9yI81enU+0+21NPgGp6Ius8glqDeqyKw zLfEwRj9hqg0vu0sZoBG0k+z0ZqbjvZf4iuruqXEd3rRzMyc0+Mj5sENg/1DGR7RvkGcJjlS LOcsaNBAHgBw1Mp5qkpFMIpfdftpQ4dRLs12Sx2Yo/SsgrmPxeulV7kAvPC0h+g70JL0/jl6 wmPJc2fHYgbgigew+3jAnOKB7h8qTx/MQXzNmwVRAgLjIncRVMwxT1QAczO9tgCrVwbC0++U tMqscs+W8mgEyok3JzY/paDazMn245bB6gQbnwdkqjTtD5cEXD2uuPlGl7hXkuqVOq6UC9f7 V3Xf4B0bY6KZrSEBeQs+HSTb6dFgWFu/sKJbFg8VHZkCMRkCgNaOXmIeGz/Izb+1CVS0Vc2Q OJE/3QcCd8ba5lHx1nxEfZbWT7jQRMvye8HRKR8CR8meNWbVP9xbGtPTzQW74skIK/MTi1z6 HAztGQARAQABiQIlBBgBCAAPBQJaEex0AhsMBQkJZgGAAAoJEKBSnOLnk4RrSRIQAKUPnTvF Ln7sK9t/9DRrXCUIdIytdqGD8RWZIPlxbhvGYY82C6MHAGQgzEukUob7wNyL2wByqrv4ZFtn 7k9aT2vZtbCO+RNwcjvMJLb3V3l6evlxbclTHO4GnRj4A5Wz9/IZYJUoK+j+Hk7huDo605xW 9xc9pxGVEdH3u6SzM2X+k2/8DtGiQ18iqBv0+T83u+6z2zZ1M6dyFk4DZV7EKrK6ujz253YP fE6Fkst9iy5SPwik+1BJKdTy/Q34fDJDveMITGoZnHbusDjzIq5kpsot6TaLqub/6jBK+jtE KJrjNEO/iBt5cb+aLN1O0VYVTtnE8xzkgg4iOkxKTuF7CBaxXEtMlILWFA8AWC+gUwtVW/c9 EXUiHRnZ7HO8KZHqqhIRTabb6PH9VUXt4nTRueDEIycCPN61jt9MdJ3vGyfxwgr4bid/+oen xWbp7+bETp3Lpqld4LTAHCQIk9/8hGTylyrVDsRj9hSu3pROukty95fu0WTas33RIcA4PdFL WQcPCXb1CrYYV/pNbhApydHxuDdMpQzbpTO8WbBox+SLIvD47LNErJepOX4vIVZpEDvo526g v+Q3PxEPdRVparjbGNj205qjIFagmQnkyjzqPgMznDxQPz9uVw+1IlF5PakH2ZohFQBqtorj BWeRBWO66DNrb/XrZSjl1bkWynWZ Message-ID: Date: Mon, 4 May 2020 08:58:07 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <87ftcgz58n.fsf@mid.deneb.enyo.de> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Content-Language: en-US X-Authenticated-Sender: nmanthey@conp-solutions.com X-Virus-Scanned: Clear (ClamAV 0.102.2/25801/Sun May 3 14:08:15 2020) X-Spam-Status: No, score=-8.1 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_STATUS, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2020 06:58:12 -0000 On 04.05.20 06:36, Florian Weimer wrote: > * Norbert Manthey: > >> This change has been tested on top of glibc 2.23 in an Ubuntu 16.04 >> environment for programs that benefit from huge pages due to their >> large memory usage and pseudo-random memory access patterns >> (e.g. SAT solvers, model checkers, optimization tools an >> others). More details on the performance improvements for these >> tools can be found in https://arxiv.org/abs/2004.14378, e.g. page 9. > Please show us the output of > > $ cat /sys/kernel/mm/transparent_hugepage/enabled > > from your test systems. Thanks. I have seen [madvise] as the default for different Ubuntu versions (16.04, 18.04 and 20.04), so I expect it to be around for at least the next 5 years. $ cat /sys/kernel/mm/transparent_hugepage/enabled always [madvise] never > > In the paper, you say this: > > | Because there might be applications running on the host that would > | suffer from larger pages, THP is usually disabled on physical > | systems and it is not advised to set the value to always. > > However, the default value for Red Hat Enterprise Linux 7 is, in fact, > "always", so it is puzzling why you see such a large benefit for > long-running processes. All used test systems have not been using [always], but [madvise]. For systems that already use [always], I expect the benefit to be much smaller. With the 2M alignment, there should be less 4K pages required, but we did not really measure the effect of only using the alignment. Best, Norbert