From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.wavenetuk.net (smtp.wavenetuk.net [195.26.36.10]) by sourceware.org (Postfix) with ESMTP id F309D385782C for ; Tue, 5 Jan 2021 14:28:33 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.3.2 sourceware.org F309D385782C Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=sandoe.co.uk Authentication-Results: sourceware.org; spf=none smtp.mailfrom=iain@sandoe.co.uk Received: from [192.168.1.212] (host81-138-1-83.in-addr.btopenworld.com [81.138.1.83]) by smtp1.wavenetuk.net (Postfix) with ESMTPA id 3A16F1201253; Tue, 5 Jan 2021 14:28:33 +0000 (GMT) Content-Type: text/plain; charset=utf-8; delsp=yes; format=flowed Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Update on shared memory coarrays From: Iain Sandoe In-Reply-To: <4020eab0-cd40-7ed5-5de0-ebb0727815f9@student.ethz.ch> Date: Tue, 5 Jan 2021 14:28:13 +0000 Cc: Thomas Koenig , Fortran List , "dhumieres.dominique@free.fr" Content-Transfer-Encoding: 8bit Message-Id: References: <0770e8fb5b4b479afc98be8cf6221cc5@free.fr> <1a63faebd65be2a76627a7873d74c0c8@free.fr> <86fa53e3-155d-1912-8b88-b0789323ee09@netcologne.de> <2d94ef13-e12e-b941-dcb0-a9a79362f2d5@netcologne.de> <6D26C802-E6E0-473F-BB17-A9B373541980@sandoe.co.uk> <00ABD0DC-235B-4A4E-B9E0-B1200AC07EAD@sandoe.co.uk> <191C9834-F907-4CF0-B652-16E04A42B7AB@sandoe.co.uk> <5683F724-2723-4EE4-AC50-83FDB598D238@sandoe.co.uk> <2007278F-63EE-427A-BFB6-A504E96D3572@sandoe.co.uk> <116ed7ff-7cdc-6324-c6c3-33fa9b1a036c@netcologne.de> <72972030-407F-4BD7-9B6C-D6937292E906@sandoe.co.uk> <816E309C-4983-4159-973D-F1A46AD8CB3C@sandoe.co.uk> <4020eab0-cd40-7ed5-5de0-ebb0727815f9@student.ethz.ch> To: =?utf-8?Q?Nicolas_K=C3=B6nig?= X-Mailer: Apple Mail (2.3273) X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, KAM_COUK, KAM_DMARC_STATUS, KAM_LAZY_DOMAIN_SECURITY, KHOP_HELO_FCRDNS, SPF_HELO_NONE, SPF_NONE, TXREP autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on server2.sourceware.org X-BeenThere: fortran@gcc.gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Fortran mailing list List-Unsubscribe: , List-Archive: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2021 14:28:35 -0000 Hi Nicolas, Nicolas König wrote: > > On 05/01/2021 14:11, Iain Sandoe wrote: >> Hi folks, >> Nicolas König wrote: >>> On 04/01/2021 03:34, Iain Sandoe wrote: >>>> Thomas Koenig via Fortran wrote: >>>>> Am 03.01.21 um 16:42 schrieb Iain Sandoe: >>>> It seems shm is not a favoured IPC mechanism for Darwin (it’s >>>> possibly disallowed for GUI apps, according to search results - I >>>> didn’t check that) >>>>>> (we can file a radar, and maybe it gets fixed at some point - but >>>>>> that’s no help to all the existing system versions) >>>>> >>>>> Makes sense to file a bug report with Apple. And yes, we'll have to >>>>> deal with this some other way. >>>> [for command line apps] Either mechanism for accessing shm works; >>>> [1] shm_open / fd / mmap / munmap >>>> [2] shmget / shmat / shmdt <= allocation is rounded up to the nearest >>>> page boundary. >>>> Both have the constraint [on Darwin] that the size of the allocation >>>> cannot be increased after it is set (by ftruncate() [1] or shmget() >>>> [2]). >>>> It can be decreased (by [2] only), but it’s not clear if that >>>> actually has any physical effect. >>> >>> That's a problem, since coarrays can be allocated at runtime, and we >>> can't possibly precompute their size. >> OK. >>> To get coarrays to work, we would probably need to create new shared >>> memory objects (with predictable names, so they can be opened from >>> other images), but that would be a significant time investment. >> How does it work between different images with shared memory unless the >> name to shm_open () is predictable between images? > > Currently, the images are launched by one "master"-process, that also > handles all the management. There, we can get some shared memory via > mmap(MAP_ANON | MAP_SHARED), so the master could communicate the base > name of the image. At the moment, it just opens a shared_memory object > and passes the file descriptor to its children. Ah, I see - so the name can in that case be arbitrary OTOH, what if one runs multiple Fortran codes (as the same user)? (some simple hashing with the the master’s PID would fix that, I guess) >>> It also doesn't bring my hopes up that other things like >>> pthread_{mutex,cond}attr_setpshared() will work. >> Darwin is Posix SUSv3 (actually, AFAICT the majority of the certified >> platforms are; there only seems to be one certified to UNIX7). >> - so pthreads stuff that’s mandated in SUSv3 is definitely present on >> Darwin. >> (in practice some of the optional stuff is also present - you can look at >> what libstdc++ does to get an idea of what works in a cross-platform >> manner) > > Process shared mutexes/condition variables are optional :) Attached is a > test case that should check whether the two work. > This will work from macOS 10.7+ (Darwin11+) earlier OS versions fail with pthread_mutexattr_setpshared(&mattr, PTHREAD_PROCESS_SHARED)failed: Undefined error: 0 (the compile succeeds, but some functionality is missing). Iain