Hi! On 2023-02-13T15:20:07+0000, Andrew Stubbs wrote: > On 13/02/2023 14:38, Thomas Schwinge wrote: >> On 2022-03-08T11:30:55+0000, Hafiz Abid Qadeer wrote: >>> From: Andrew Stubbs >>> >>> Add a new option. It will be used in follow-up patches. >> >>> --- a/gcc/doc/invoke.texi >>> +++ b/gcc/doc/invoke.texi >> >>> +@option{-foffload-memory=pinned} forces all host memory to be pinned (this >>> +mode may require the user to increase the ulimit setting for locked memory). >> >> So, this is currently implemented via 'mlockall', which, as discussed, >> (a) has issues ('ulimit -l'), and (b) doesn't actually achieve what it >> meant to achieve (because it doesn't register the page-locked memory with >> the GPU driver). >> [...] >> As '-foffload-memory=pinned', per the name >> of the option, concerns itself with memory used in offloading but not >> host execution generally, why are we actually attempting to "[force] all >> host memory to be pinned" -- why not just the memory that's being used >> with offloading? That is, if '-foffload-memory=pinned' is set, register >> as page-locked with the GPU driver all memory that appears in OMP >> offloading data regions, such as OpenMP 'target' 'map' clauses etc. That >> way, this is directed at the offloading data transfers, as itended, but >> at the same time we don't "waste" page-locked memory for generic host >> memory allocations. What do you think -- you, who've spent a lot more >> time on this topic than I have, so it's likely possible that I fail to >> realize some "details"? > > The main reason it is the way it is is because in general it's not > possible to know what memory is going to be offloaded at the time it is > allocated (and stack/static memory is never allocated that way). > > If there's a way to pin it after the fact then maybe that's not a > terrible idea? [...] I've now pushed to devel/omp/gcc-12 branch my take on this in commit 43095690ea519205bf56fc148b346edaa43e0f0f "'-foffload-memory=pinned' using offloading device interfaces", and for changes related to og12 commit 15d0f61a7fecdc8fd12857c40879ea3730f6d99f "Merge non-contiguous array support patches": commit 694bbd399c1323975b4a6735646e46c6914de63d "'-foffload-memory=pinned' using offloading device interfaces for non-contiguous array support", see attached. Grüße Thomas ----------------- Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955