From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by sourceware.org (Postfix) with ESMTPS id 653EC383F87E for ; Mon, 4 May 2020 13:31:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org 653EC383F87E Received: by mail-qt1-x842.google.com with SMTP id h26so13904322qtu.8 for ; Mon, 04 May 2020 06:31:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:references:from:autocrypt:subject:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=Cg8eoPeZX3s9+9N0VOXdkKMpPdv1OKvcEElnYsihRo0=; b=IYvb6KNgylUaRY5OolKXi/wCYdYfKZPB9q/HDoTLifocIpMWQdvQU9g4ns3PwUEls9 i2ZMTRbdxCkJCLQwLsdxPsqzkxRVpUc8ddY5nuLFObR6Fx7o9/CVEJjw0Lau8nM6R9nA X6J10gc0SxPRYe4DQxXUb8J8wj2O62rtAQCbs2QC05lL9xyBmG3O0tvTqotyGL8FB7xV IIi6y5Tv5XWr4njvutyFRQSX/Nor4TmPf1AoTd4w2c4R3D8JgALcK4RQVFp0qjHnsnPT dB7MBCayFJE+DjcyGt8LJyWrIhyh+XHNrpSv43ny3IsHwgGYhB42u6VFOZTWyfNuKh3Y Iqig== X-Gm-Message-State: AGi0PuY/xKDb9PsIZhDDogmE7ucfEBsnv1Jrc7egpnX0LMtQdvfQrPuO 9j1f8v3WqF1DutVguXk2WDPcaIInaOU= X-Google-Smtp-Source: APiQypKhTl5FQ5zJVnVoHEKLIJk+eXt/Uj47962PvOf6wkHl9Z5AuY/4oXRBuhvgXgRU58LN1h3vsA== X-Received: by 2002:ac8:5256:: with SMTP id y22mr15975245qtn.321.1588599092692; Mon, 04 May 2020 06:31:32 -0700 (PDT) Received: from [192.168.1.4] ([177.194.48.209]) by smtp.googlemail.com with ESMTPSA id w27sm11060141qtc.18.2020.05.04.06.31.31 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 04 May 2020 06:31:32 -0700 (PDT) To: libc-alpha@sourceware.org References: <7bd30e6c-7ae8-4c03-a818-6309351b3df9@email.android.com> <877dxsw1eu.fsf@mid.deneb.enyo.de> From: Adhemerval Zanella Autocrypt: addr=adhemerval.zanella@linaro.org; prefer-encrypt=mutual; keydata= mQINBFcVGkoBEADiQU2x/cBBmAVf5C2d1xgz6zCnlCefbqaflUBw4hB/bEME40QsrVzWZ5Nq 8kxkEczZzAOKkkvv4pRVLlLn/zDtFXhlcvQRJ3yFMGqzBjofucOrmdYkOGo0uCaoJKPT186L NWp53SACXguFJpnw4ODI64ziInzXQs/rUJqrFoVIlrPDmNv/LUv1OVPKz20ETjgfpg8MNwG6 iMizMefCl+RbtXbIEZ3TE/IaDT/jcOirjv96lBKrc/pAL0h/O71Kwbbp43fimW80GhjiaN2y WGByepnkAVP7FyNarhdDpJhoDmUk9yfwNuIuESaCQtfd3vgKKuo6grcKZ8bHy7IXX1XJj2X/ BgRVhVgMHAnDPFIkXtP+SiarkUaLjGzCz7XkUn4XAGDskBNfbizFqYUQCaL2FdbW3DeZqNIa nSzKAZK7Dm9+0VVSRZXP89w71Y7JUV56xL/PlOE+YKKFdEw+gQjQi0e+DZILAtFjJLoCrkEX w4LluMhYX/X8XP6/C3xW0yOZhvHYyn72sV4yJ1uyc/qz3OY32CRy+bwPzAMAkhdwcORA3JPb kPTlimhQqVgvca8m+MQ/JFZ6D+K7QPyvEv7bQ7M+IzFmTkOCwCJ3xqOD6GjX3aphk8Sr0dq3 4Awlf5xFDAG8dn8Uuutb7naGBd/fEv6t8dfkNyzj6yvc4jpVxwARAQABtElBZGhlbWVydmFs IFphbmVsbGEgTmV0dG8gKExpbmFybyBWUE4gS2V5KSA8YWRoZW1lcnZhbC56YW5lbGxhQGxp bmFyby5vcmc+iQI3BBMBCAAhBQJXFRpKAhsDBQsJCAcDBRUKCQgLBRYCAwEAAh4BAheAAAoJ EKqx7BSnlIjv0e8P/1YOYoNkvJ+AJcNUaM5a2SA9oAKjSJ/M/EN4Id5Ow41ZJS4lUA0apSXW NjQg3VeVc2RiHab2LIB4MxdJhaWTuzfLkYnBeoy4u6njYcaoSwf3g9dSsvsl3mhtuzm6aXFH /Qsauav77enJh99tI4T+58rp0EuLhDsQbnBic/ukYNv7sQV8dy9KxA54yLnYUFqH6pfH8Lly sTVAMyi5Fg5O5/hVV+Z0Kpr+ZocC1YFJkTsNLAW5EIYSP9ftniqaVsim7MNmodv/zqK0IyDB GLLH1kjhvb5+6ySGlWbMTomt/or/uvMgulz0bRS+LUyOmlfXDdT+t38VPKBBVwFMarNuREU2 69M3a3jdTfScboDd2ck1u7l+QbaGoHZQ8ZNUrzgObltjohiIsazqkgYDQzXIMrD9H19E+8fw kCNUlXxjEgH/Kg8DlpoYJXSJCX0fjMWfXywL6ZXc2xyG/hbl5hvsLNmqDpLpc1CfKcA0BkK+ k8R57fr91mTCppSwwKJYO9T+8J+o4ho/CJnK/jBy1pWKMYJPvvrpdBCWq3MfzVpXYdahRKHI ypk8m4QlRlbOXWJ3TDd/SKNfSSrWgwRSg7XCjSlR7PNzNFXTULLB34sZhjrN6Q8NQZsZnMNs TX8nlGOVrKolnQPjKCLwCyu8PhllU8OwbSMKskcD1PSkG6h3r0AquQINBFcVGkoBEACgAdbR Ck+fsfOVwT8zowMiL3l9a2DP3Eeak23ifdZG+8Avb/SImpv0UMSbRfnw/N81IWwlbjkjbGTu oT37iZHLRwYUFmA8fZX0wNDNKQUUTjN6XalJmvhdz9l71H3WnE0wneEM5ahu5V1L1utUWTyh VUwzX1lwJeV3vyrNgI1kYOaeuNVvq7npNR6t6XxEpqPsNc6O77I12XELic2+36YibyqlTJIQ V1SZEbIy26AbC2zH9WqaKyGyQnr/IPbTJ2Lv0dM3RaXoVf+CeK7gB2B+w1hZummD21c1Laua +VIMPCUQ+EM8W9EtX+0iJXxI+wsztLT6vltQcm+5Q7tY+HFUucizJkAOAz98YFucwKefbkTp eKvCfCwiM1bGatZEFFKIlvJ2QNMQNiUrqJBlW9nZp/k7pbG3oStOjvawD9ZbP9e0fnlWJIsj 6c7pX354Yi7kxIk/6gREidHLLqEb/otuwt1aoMPg97iUgDV5mlNef77lWE8vxmlY0FBWIXuZ yv0XYxf1WF6dRizwFFbxvUZzIJp3spAao7jLsQj1DbD2s5+S1BW09A0mI/1DjB6EhNN+4bDB SJCOv/ReK3tFJXuj/HbyDrOdoMt8aIFbe7YFLEExHpSk+HgN05Lg5TyTro8oW7TSMTk+8a5M kzaH4UGXTTBDP/g5cfL3RFPl79ubXwARAQABiQIfBBgBCAAJBQJXFRpKAhsMAAoJEKqx7BSn lIjvI/8P/jg0jl4Tbvg3B5kT6PxJOXHYu9OoyaHLcay6Cd+ZrOd1VQQCbOcgLFbf4Yr+rE9l mYsY67AUgq2QKmVVbn9pjvGsEaz8UmfDnz5epUhDxC6yRRvY4hreMXZhPZ1pbMa6A0a/WOSt AgFj5V6Z4dXGTM/lNManr0HjXxbUYv2WfbNt3/07Db9T+GZkpUotC6iknsTA4rJi6u2ls0W9 1UIvW4o01vb4nZRCj4rni0g6eWoQCGoVDk/xFfy7ZliR5B+3Z3EWRJcQskip/QAHjbLa3pml xAZ484fVxgeESOoaeC9TiBIp0NfH8akWOI0HpBCiBD5xaCTvR7ujUWMvhsX2n881r/hNlR9g fcE6q00qHSPAEgGr1bnFv74/1vbKtjeXLCcRKk3Ulw0bY1OoDxWQr86T2fZGJ/HIZuVVBf3+ gaYJF92GXFynHnea14nFFuFgOni0Mi1zDxYH/8yGGBXvo14KWd8JOW0NJPaCDFJkdS5hu0VY 7vJwKcyHJGxsCLU+Et0mryX8qZwqibJIzu7kUJQdQDljbRPDFd/xmGUFCQiQAncSilYOcxNU EMVCXPAQTteqkvA+gNqSaK1NM9tY0eQ4iJpo+aoX8HAcn4sZzt2pfUB9vQMTBJ2d4+m/qO6+ cFTAceXmIoFsN8+gFN3i8Is3u12u8xGudcBPvpoy4OoG Subject: Re: [[RFC][PATCH] v1 0/2] malloc/realloc with transparent huge page support Message-ID: <2ccb615d-7c3d-d206-a507-8971686a7b5a@linaro.org> Date: Mon, 4 May 2020 10:31:29 -0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 MIME-Version: 1.0 In-Reply-To: <877dxsw1eu.fsf@mid.deneb.enyo.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF, RCVD_IN_DNSWL_NONE, SPF_HELO_NONE, SPF_PASS, TXREP autolearn=ham autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: libc-alpha@sourceware.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Libc-alpha mailing list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 May 2020 13:31:37 -0000 On 04/05/2020 05:27, Florian Weimer wrote: > * nmanthey: > >>> Still, it is easier to change a run-time kernel setting than to >>> upgrade glibc. >> >> That is something I do not have control over. Hence, I would offer >> the ability to running processes. Statically linked binaries could >> come with their own modified glibc; and for such a purpose these >> patches are valuable. There might also be scenarios where not all >> processes should get huge pages, so being able to enable this on a >> per-process level looks reasonable to me. > > I'm not sure if it is a good idea to work around kernel problems in > glibc just because it happens to work in your setup, given your local > constraints. I agree and coupling the hugepages with libc memory allocation algorithm usually results in way more code complexity which I am not sure it pays out. It is either a configure option that defines a static hugepage size (jemalloc); or try to use an special filesystem (tcmalloc which also requires environment variables), or extend the tooling and dynamic linker to incorporate the hugepage more natural (AIX for instance, where static linker may set some DT_LOAD and dynamic allocation to use hugepages). In any case, this is orthogonal to what Linux kernel is moving forward which is to hide all these details from the programs with THP. The paper also has a bold claim which I am not sure it is exactly true: "Globally enabling transparent huge pages for all running applications on the system is usually forbidden both in industry and academia by administrators due to a variety of potential side effects, which might slow down a variety of programs." > > It may be time for per-namespace/cgroups settings for hugepages, > though. I think this could be a better approach indeed. > >>> I suggest to repeat the experiment with "always". There is a reason >>> why this setting exists. The results presented so far are incomplete. >>> >>> The paper doesn't provide details on the NUMA configuration of the >>> test system, so one has to wonder if there are any surprises there as >>> well. >> >> I agree, and I can look up the numa configuration. Even with less >> performance increase, as a unprivileged user, I would like to enable >> THP. >> >> Do you have a specific ask of what else should be measured? > > I'm not a virtual memory management or performance expert. I will ask > around. > > The immediate problem I see is that requesting transparent hugepages > in this way forces the kernel to perform more aggressive > defragmentation, which can hurt overall system performance if > processes are shortlived. >