Malloc NUMA Aware

Malloc NUMA Aware

[Nicholas Krause]

Greetings,
With Ryzen and other microprocessors being more NUMA based in design due to
chiplets e.t.c., I curious as to the support for NUMA aware allocations 
in glibc.

Thanks,
Nick

Re: Malloc NUMA Aware

[Carlos O'Donell]

On Thu, Jan 2, 2020 at 11:18 PM Nicholas Krause <xerofoify@gmail.com> wrote:
> With Ryzen and other microprocessors being more NUMA based in design due to
> chiplets e.t.c., I curious as to the support for NUMA aware allocations
> in glibc.

NUMA awareness means different things to different people.

(1) A general purpose allocator that is optimal on NUMA systems?

(2) A new API that allows finer grained control over the location of
the memory requested within the hierarchy?

In general glibc's malloc binds a thread to a given arena (collection
of heaps) and attempts to avoid moving the thread during its lifetime.
This behaviour should be helpful on NUMA systems. This resistance to
moving the threads should mean that a thread's own allocations are
more likely to come from a local node and perform well (data
locality). In summary we are aware that there could be NUMA problems,
we think about them as we redesign and refine glibc's malloc, but we
haven't see any reported bad performance. Likewise there are issues to
consider with transparent huge pages (THP).

When using a producer consumer model with threads on different nodes
this becomes problematic. No matter which node you allocate from, if
your producer and consumer threads are on different nodes, then this
will cause performance degradation. Therefore it's often better to pin
such threads on a single node given knowledge of the system hierarchy.

There is a lot of discussion about this around the use of OpenMP on
large >1024 CPU systems. There is also a discussion about a NUMA-aware
allocator for OpenMP as part of the standard.

The only NUMA-aware allocator that I know about with new API's is
being developed by Intel as memkind
(https://github.com/memkind/memkind) but this is completely different
from the current glibc malloc which is a general-purpose system
allocator.

Does that answer your questions?

Cheers,
Carlos.

Re: Malloc NUMA Aware

[Nicholas Krause]

On 1/4/20 9:54 AM, Carlos O'Donell wrote:
> On Thu, Jan 2, 2020 at 11:18 PM Nicholas Krause <xerofoify@gmail.com> wrote:
>> With Ryzen and other microprocessors being more NUMA based in design due to
>> chiplets e.t.c., I curious as to the support for NUMA aware allocations
>> in glibc.
> NUMA awareness means different things to different people.
>
> (1) A general purpose allocator that is optimal on NUMA systems?
>
> (2) A new API that allows finer grained control over the location of
> the memory requested within the hierarchy?
>
> In general glibc's malloc binds a thread to a given arena (collection
> of heaps) and attempts to avoid moving the thread during its lifetime.
> This behaviour should be helpful on NUMA systems. This resistance to
> moving the threads should mean that a thread's own allocations are
> more likely to come from a local node and perform well (data
> locality). In summary we are aware that there could be NUMA problems,
> we think about them as we redesign and refine glibc's malloc, but we
> haven't see any reported bad performance. Likewise there are issues to
> consider with transparent huge pages (THP).
>
> When using a producer consumer model with threads on different nodes
> this becomes problematic. No matter which node you allocate from, if
> your producer and consumer threads are on different nodes, then this
> will cause performance degradation. Therefore it's often better to pin
> such threads on a single node given knowledge of the system hierarchy.
>
> There is a lot of discussion about this around the use of OpenMP on
> large >1024 CPU systems. There is also a discussion about a NUMA-aware
> allocator for OpenMP as part of the standard.
>
> The only NUMA-aware allocator that I know about with new API's is
> being developed by Intel as memkind
> (https://github.com/memkind/memkind) but this is completely different
> from the current glibc malloc which is a general-purpose system
> allocator.
>
> Does that answer your questions?
>
> Cheers,
> Carlos.
I wasn't sure how much the areas helped so that does answer my
question.
Nick