Re: malloc: Optimize the number of arenas for better application performance

[Siddhesh Poyarekar <siddhesh@sourceware.org>]

On 28/06/2022 16:48, Florian Weimer wrote:
> * Yang Yanchao via Libc-alpha:
> 
>> At Kunpeng920 platform, tpcc-mysql scores decreased by about 11.2%
>> between glibc-2.36 and glibc2.28.
>> Comparing the code, I find that the two commits causes performance
>> degradation.
>> 11a02b035b46 (misc: Add __get_nprocs_sched)
>> 97ba273b5057 (linux: __get_nprocs_sched: do not feed CPU_COUNT_S with
>> garbage [BZ #28850])
>>
>> These two patches modify the default behavior.
>> However, my machine is 96 cores and I have 91 cores bound.
>> It means that perhaps the current way of computing arenas is not optimal.
>> So I roll back some of the code submitted by 11a02b035(misc: Add
>> __get_nprocs_sched).
>>
>> ---
>>    malloc/arena.c | 2 +-
>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/malloc/arena.c b/malloc/arena.c
>> index 0a684a720d..a1ee7928d3 100644
>> --- a/malloc/arena.c
>> +++ b/malloc/arena.c
>> @@ -937,7 +937,7 @@ arena_get2 (size_t size, mstate avoid_arena)
>>                narenas_limit = mp_.arena_max;
>>              else if (narenas > mp_.arena_test)
>>                {
>> -              int n = __get_nprocs_sched ();
>> +              int n = __get_nprocs ();
>>
>>                  if (n >= 1)
>>                    narenas_limit = NARENAS_FROM_NCORES (n);
> 
> How many threads does tpcc-mysql create?
> 
> I wonder if all threads get their own arena with the larger count, and
> there is some arena sharing with the smaller count.

A simple test to determine this could be to repeat the test with 
different values of arena_max to see if there's a trend.

Siddhesh