Hi folks, Nicolas König wrote: > On 04/01/2021 03:34, Iain Sandoe wrote: >> Thomas Koenig via Fortran wrote: >>> Am 03.01.21 um 16:42 schrieb Iain Sandoe: >> It seems shm is not a favoured IPC mechanism for Darwin (it’s possibly >> disallowed for GUI apps, according to search results - I didn’t check >> that) >>>> (we can file a radar, and maybe it gets fixed at some point - but >>>> that’s no help to all the existing system versions) >>> >>> Makes sense to file a bug report with Apple. And yes, we'll have to >>> deal with this some other way. >> [for command line apps] Either mechanism for accessing shm works; >> [1] shm_open / fd / mmap / munmap >> [2] shmget / shmat / shmdt <= allocation is rounded up to the nearest >> page boundary. >> Both have the constraint [on Darwin] that the size of the allocation >> cannot be increased after it is set (by ftruncate() [1] or shmget() [2]). >> It can be decreased (by [2] only), but it’s not clear if that actually >> has any physical effect. > > That's a problem, since coarrays can be allocated at runtime, and we > can't possibly precompute their size. OK. > To get coarrays to work, we would probably need to create new shared > memory objects (with predictable names, so they can be opened from other > images), but that would be a significant time investment. How does it work between different images with shared memory unless the name to shm_open () is predictable between images? (there has to be some shared key of some form, right?) > It also doesn't bring my hopes up that other things like > pthread_{mutex,cond}attr_setpshared() will work. Darwin is Posix SUSv3 (actually, AFAICT the majority of the certified platforms are; there only seems to be one certified to UNIX7). - so pthreads stuff that’s mandated in SUSv3 is definitely present on Darwin. (in practice some of the optional stuff is also present - you can look at what libstdc++ does to get an idea of what works in a cross-platform manner) > I think it's probably better to focus the attention on getting it to work > on Linux/BSD. Well, I can understand making progress in that way - but it’s a bit of a slippery slope towards an “x86 Linux only” compiler (if we keep taking that route until a significant portion of the useful features don’t work elsewhere). Of course, this particular case is not so exclusive [it appears from previous mails that Darwin is the only one with an issue], but as a general rule it would be good to pick an impl that works on at least the regularly used/tested targets. >> I guess that one ought to consider that a change in allocation could >> trigger a copy on any system (since there might not be contiguous memory >> pages available for an arbitrary increase) - so a strategy which either >> allocates separae fragments for each need - or waits until the size is >> known before allocation is likely to perform better on average, > > There's no need to have the continuous virtual memory backed by > continuous physical memory, so nothing here should trigger a relocation > of the actual physical memory. There are just some entries in the page > table pointing to the same pages. I guess if everything is done on page boundaries & sizes, that can work - providing one doesn’t mind the base pointer changing across resizes. ==== Update: three codes attached. 1. Posix shm implementation; works fine but is not resizeable. 2. Darwin shm implementation : works fine but is only resizable downwards and it’s not clear if the resources are actually released. 3. mmapped shared file : This works [on Darwin at least] and is resizable. I would assume that, in reality, all accesses would be to the buffer cache [unless something actually forces the cache to be flushed] and therefore that the performance ought to be similar to shm. Worst case, one could point /tmp at a ramdisk .. (I tend to do that anyway, I don’t want my sshds getting hammered by toolchain builds). Thoughts? Iain