Re: [[RFC][PATCH] v1 0/2] malloc/realloc with transparent huge page support

[Florian Weimer <fw@deneb.enyo.de>]

* nmanthey:

>> Still, it is easier to change a run-time kernel setting than to 
>> upgrade glibc. 
>
> That is something I do not have control over. Hence, I would offer
> the ability to running processes. Statically linked binaries could
> come with their own modified glibc; and for such a purpose these
> patches are valuable. There might also be scenarios where not all
> processes should get huge pages, so being able to enable this on a
> per-process level looks reasonable to me.

I'm not sure if it is a good idea to work around kernel problems in
glibc just because it happens to work in your setup, given your local
constraints.

It may be time for per-namespace/cgroups settings for hugepages,
though.

>> I suggest to repeat the experiment with "always".  There is a reason 
>> why this setting exists.  The results presented so far are incomplete. 
>>
>> The paper doesn't provide details on the NUMA configuration of the 
>> test system, so one has to wonder if there are any surprises there as 
>> well. 
>
> I agree, and I can look up the numa configuration. Even with less
> performance increase, as a unprivileged user, I would like to enable
> THP.
>
> Do you have a specific ask of what else should be measured?

I'm not a virtual memory management or performance expert.  I will ask
around.

The immediate problem I see is that requesting transparent hugepages
in this way forces the kernel to perform more aggressive
defragmentation, which can hurt overall system performance if
processes are shortlived.