Re: [PATCH] libgomp, openmp: pinned memory

[Andrew Stubbs <ams@codesourcery.com>]

On 10/02/2023 15:11, Thomas Schwinge wrote:
> Hi!
> 
> Re OpenMP 'pinned' memory allocator trait semantics vs. 'omp_realloc':
> 
> On 2022-01-13T13:53:03+0000, Andrew Stubbs <ams@codesourcery.com> wrote:
>> On 05/01/2022 17:07, Andrew Stubbs wrote:
>>> [...], I'm working on an implementation using mmap instead of malloc
>>> for pinned allocations.  [...]
> 
>> This means that large allocations will now be page aligned and therefore
>> pin the smallest number of pages for the size requested, and that that
>> memory will be unpinned automatically when freed via munmap, or moved
>> via mremap.
> 
>> --- /dev/null
>> +++ b/libgomp/config/linux/allocator.c
> 
>> +static void *
>> +linux_memspace_realloc (omp_memspace_handle_t memspace, void *addr,
>> +                     size_t oldsize, size_t size, int oldpin, int pin)
>> +{
>> +  if (oldpin && pin)
>> +    {
>> +      void *newaddr = mremap (addr, oldsize, size, MREMAP_MAYMOVE);
>> +      if (newaddr == MAP_FAILED)
>> +     return NULL;
>> +
>> +      return newaddr;
>> +    }
>> +  else if (oldpin || pin)
>> +    {
>> +      void *newaddr = linux_memspace_alloc (memspace, size, pin);
>> +      if (newaddr)
>> +     {
>> +       memcpy (newaddr, addr, oldsize < size ? oldsize : size);
>> +       linux_memspace_free (memspace, addr, oldsize, oldpin);
>> +     }
>> +
>> +      return newaddr;
>> +    }
>> +  else
>> +    return realloc (addr, size);
>> +}
> 
> I did wonder if 'mremap' with 'MREMAP_MAYMOVE' is really acceptable here,
> given OpenMP 5.2, 6.2 "Memory Allocators": "Allocators with the 'pinned'
> trait defined to be 'true' ensure that their allocations remain in the
> same storage resource at the same location for their entire lifetime."
> I'd have read into this that 'realloc' may shrink or enlarge the region
> (unless even that considered faulty), but the region must not be moved
> ("same location"), thus no 'MREMAP_MAYMOVE'; see 'man 2 mremap'

I don't think the OpenMP specification really means that any program 
using omp_realloc should abort randomly depending on the vagaries of 
chaos? What are we supposed to do? Hugely over-allocate in case realloc 
is ever called?

Andrew