From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from crocodile.elm.relay.mailchannels.net (crocodile.elm.relay.mailchannels.net [23.83.212.45]) by sourceware.org (Postfix) with ESMTPS id 319E23858438 for ; Thu, 19 Aug 2021 11:49:05 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.1 sourceware.org 319E23858438 X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 613A010298A; Thu, 19 Aug 2021 11:49:03 +0000 (UTC) Received: from pdx1-sub0-mail-a83.g.dreamhost.com (100-101-162-86.trex.outbound.svc.cluster.local [100.101.162.86]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 8D9001028A0; Thu, 19 Aug 2021 11:49:02 +0000 (UTC) X-Sender-Id: dreamhost|x-authsender|siddhesh@gotplt.org Received: from pdx1-sub0-mail-a83.g.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384) by 100.101.162.86 (trex/6.3.3); Thu, 19 Aug 2021 11:49:03 +0000 X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|siddhesh@gotplt.org X-MailChannels-Auth-Id: dreamhost X-Stupid-Drop: 7a6ad4aa0c199729_1629373742917_2798963192 X-MC-Loop-Signature: 1629373742917:3216659124 X-MC-Ingress-Time: 1629373742917 Received: from pdx1-sub0-mail-a83.g.dreamhost.com (localhost [127.0.0.1]) by pdx1-sub0-mail-a83.g.dreamhost.com (Postfix) with ESMTP id 127AB7F700; Thu, 19 Aug 2021 04:49:02 -0700 (PDT) Received: from [192.168.1.165] (unknown [1.186.101.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: siddhesh@gotplt.org) by pdx1-sub0-mail-a83.g.dreamhost.com (Postfix) with ESMTPSA id 490C17F6EA; Thu, 19 Aug 2021 04:48:57 -0700 (PDT) Subject: Re: [PATCH v2 0/4] malloc: Improve Huge Page support To: Adhemerval Zanella , libc-alpha@sourceware.org Cc: Norbert Manthey , Guillaume Morin References: <20210818142000.128752-1-adhemerval.zanella@linaro.org> <5e37cb66-fd93-5d27-ec7b-28f7cf636246@linaro.org> X-DH-BACKEND: pdx1-sub0-mail-a83 From: Siddhesh Poyarekar Message-ID: <9c13a602-573a-666f-071c-f88c1f857b5c@sourceware.org> Date: Thu, 19 Aug 2021 17:18:53 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <5e37cb66-fd93-5d27-ec7b-28f7cf636246@linaro.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-3485.7 required=5.0 tests=BAYES_00, JMQ_SPF_NEUTRAL, KAM_DMARC_NONE, KAM_DMARC_STATUS, NICE_REPLY_A, RCVD_IN_BARRACUDACENTRAL, RCVD_IN_BL_SPAMCOP_NET, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, SPF_HELO_NONE, SPF_NEUTRAL, TXREP autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Aug 2021 11:49:08 -0000 On 8/19/21 4:56 PM, Adhemerval Zanella wrote: > I though about it, and decided to use two tunables because although > for mmap() system allocation both tunable are mutually exclusive > (since it does not make sense to madvise() a mmap(MAP_HUGETLB) > we still use sbrk() on main arena. The way I did for sbrk() is to align > to the THP page size advertisen by the kernel, so using the tunable > does change the behavior slightly (it is not 'transparent' as the > madvise call). >=20 > So to use only one tunable would require to either drop the sbrk() > madvise when MAP_HUGETLB is used, move it to another tunable (say > '3: HugeTLB enabled with default hugepage size and madvise() on sbrk())= , > or assume it when huge pages should be used. >=20 > (and how do we handle sbrk() with explicit size?) > > If one tunable is preferable I think it would be something like: >=20 > 0: Disabled (default) > 1: Transparent, where we emulate "always" behaviour of THP > sbrk() is also aligned to huge page size and issued madvise() > 2: HugeTLB enabled with default hugepage size and sbrk() as > handled are 1 >> : HugeTLB enabled with the specified page size and sbrk() > are handled as 1 >=20 > By forcing the sbrk() and madvise() on all tunables value make > the expectation to use huge pages in all possible occasions. What do you think about using mmap instead of sbrk for (2) and if=20 hugetlb is requested? It kinda emulates what libhugetlbfs does and=20 makes the behaviour more consistent with what is advertised by the tunabl= es. >> A simple test like below in benchtests would be very useful to at leas= t get an initial understanding of the behaviour differences with differen= t tunable values.=C2=A0 Later those who care can add more relevant worklo= ads. >=20 > Yeah, I am open to suggestions on how to properly test it. The issue > is we need to have specific system configuration either by proper > kernel support (THP) or with reserved large pages to actually test > it. >=20 > For THP the issue is really 'transparent' for user, which means that > we will need to poke on specific Linux sysfs information to check if > huge pages are being used. And we might not get the expected answer > depending of the system load and memory utilization (the advised > pages might not be moved to large pages if there is no sufficient > memory). For benchmarking we can make a minimal assumption that the user will set=20 the system up to appropriately isolate the benchmarks. As for the sysfs=20 setup, we can always test and bail if unsupported. >> You could add tests similar to mcheck and malloc-check, i.e. add $(tes= ts-hugepages) to run all malloc tests again with the various tunable valu= es.=C2=A0 See tests-mcheck for example. >=20 > Ok, I can work with this. This might not add much if the system is > not configured with either THP or with some huge page pool but at > least adds some coverage. Yeah the main intent is to simply ensure that there are no differences=20 in behaviour with hugepages. Siddhesh