From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from esa3.mentor.iphmx.com (esa3.mentor.iphmx.com [68.232.137.180]) by sourceware.org (Postfix) with ESMTPS id 0A0E03858C39 for ; Fri, 13 Jan 2023 17:52:42 +0000 (GMT) DMARC-Filter: OpenDMARC Filter v1.4.2 sourceware.org 0A0E03858C39 Authentication-Results: sourceware.org; dmarc=none (p=none dis=none) header.from=codesourcery.com Authentication-Results: sourceware.org; spf=pass smtp.mailfrom=mentor.com X-IronPort-AV: E=Sophos;i="5.97,214,1669104000"; d="scan'208";a="93247036" Received: from orw-gwy-02-in.mentorg.com ([192.94.38.167]) by esa3.mentor.iphmx.com with ESMTP; 13 Jan 2023 09:52:40 -0800 IronPort-SDR: Gni5gI2FlHNeScJu93cf8rW+plSLP0/Dw8SM/HyO1iv0mQjdDJ5UZ1zY3FTqZ02meLxvkmkOlJ y46QW9nA8J77Juyvwv4N1inzXBR95XZ80xagYoTUSfECFRr7iqW5pZRiP4ZUcNKQzejnHmtjz3 nUQy+xU7aefUw2Dv/qb+BXLvzq/Nnqm851wp8AShLfdYaK5TaRErwEuMGwcI8KbIpuoDWQc1RN o5rmhwF2qX7rbJlBHnL1rhmX7v6uTKZBqdD202HDX7/tNvX+v8zdOxcGWtnYF+QgDAk3rMfgjL LfM= Content-Type: multipart/mixed; boundary="------------WhkBAh0eSaSrntIm9lE3EkTS" Message-ID: Date: Fri, 13 Jan 2023 17:52:34 +0000 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [OG12][committed] amdgcn, libgomp: custom USM allocator From: Andrew Stubbs To: "gcc-patches@gcc.gnu.org" References: Content-Language: en-GB In-Reply-To: X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: svr-ies-mbx-12.mgc.mentorg.com (139.181.222.12) To svr-ies-mbx-11.mgc.mentorg.com (139.181.222.11) X-Spam-Status: No, score=-12.5 required=5.0 tests=BAYES_00,GIT_PATCH_0,HEADER_FROM_DIFFERENT_DOMAINS,KAM_DMARC_STATUS,NICE_REPLY_A,SPF_HELO_PASS,SPF_PASS,TXREP autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on server2.sourceware.org List-Id: --------------WhkBAh0eSaSrntIm9lE3EkTS Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit I changed it to use 128-byte alignment to match the GPU cache-lines. Committed to OG12. Andrew On 11/01/2023 18:05, Andrew Stubbs wrote: > This patch fixes a runtime issue I encountered with the AMD GCN Unified > Shared Memory implementation. > > We were using regular malloc'd memory configured into USM mode, but > there were random intermittent crashes. I can't be completely sure, but > my best guess is that the HSA driver is using malloc internally from the > same heap, and therefore using memory on the same page as the offload > kernel. What I do know is that I could make the crashes go away by > simply padding the USM allocations before and after. > > With this patch USM allocations are now completely separated from the > system heap. The custom allocator is probably less optimal is some > use-cases, but does have the advantage that all the metadata is stored > in a side-table that won't ever cause any pages to migrate back to > main-memory unnecessarily. It's still possible for the user program to > use USM memory in a way that causes it to thrash, and this might have > been the ultimate cause of the crashes, but there's not much we can do > about that here. > > I've broken the allocator out into a new file because I anticipate it > being needed in more than one place, but I didn't put full > data-isolation on it yet. > > I'll rebase, merge, and repost all of the OpenMP memory patches sometime > soonish. > > Andrew --------------WhkBAh0eSaSrntIm9lE3EkTS Content-Type: text/plain; charset="UTF-8"; name="230113-usm-align-128.patch" Content-Disposition: attachment; filename="230113-usm-align-128.patch" Content-Transfer-Encoding: base64 bGliZ29tcCwgYW1kZ2NuOiBTd2l0Y2ggVVNNIHRvIDEyOC1ieXRlIGFsaWdubWVudAoKVGhp cyBzaG91bGQgb3B0aW1pemUgY2FjaGUtbGluZXMgb24gdGhlIEFNRCBHUFVzIHNvbWV3aGF0 LgoKbGliZ29tcC9DaGFuZ2VMb2c6CgoJKiB1c20tYWxsb2NhdG9yLmMgKEFMSUdOKTogVXNl IDEyOC1ieXRlIGFsaWdubWVudC4KCmRpZmYgLS1naXQgYS9saWJnb21wL3VzbS1hbGxvY2F0 b3IuYyBiL2xpYmdvbXAvdXNtLWFsbG9jYXRvci5jCmluZGV4IGM0NTEwOTE2OWNhLi42OGMx ZWJhZmVjMiAxMDA2NDQKLS0tIGEvbGliZ29tcC91c20tYWxsb2NhdG9yLmMKKysrIGIvbGli Z29tcC91c20tYWxsb2NhdG9yLmMKQEAgLTU3LDcgKzU3LDggQEAgc3RhdGljIGludCB1c21f bG9jayA9IDA7CiBzdGF0aWMgc3RydWN0IHVzbV9zcGxheV90cmVlX3MgdXNtX2FsbG9jYXRp b25zID0geyBOVUxMIH07CiBzdGF0aWMgc3RydWN0IHVzbV9zcGxheV90cmVlX3MgdXNtX2Zy ZWVfc3BhY2UgPSB7IE5VTEwgfTsKIAotI2RlZmluZSBBTElHTihWQVIpICgoKFZBUikgKyA3 KSAmIH43KSAgICAvKiA4LWJ5dGUgZ3JhbnVsYXJpdHkuICAqLworLyogMTI4LWJ5dGUgZ3Jh bnVsYXJpdHkgbWVhbnMgR1BVIGNhY2hlLWxpbmUgYWxpZ25lZC4gICovCisjZGVmaW5lIEFM SUdOKFZBUikgKCgoVkFSKSArIDEyNykgJiB+MTI3KQogCiAvKiBDb2FsZXNjZSBjb250aWd1 b3VzIGZyZWUgc3BhY2UgaW50byBvbmUgZW50cnkuICBUaGlzIGNvbnNpZGVycyB0aGUgZW50 cmllcwogICAgZWl0aGVyIHNpZGUgb2YgdGhlIHJvb3Qgbm9kZSBvbmx5LCBzbyBpdCBzaG91 bGQgYmUgY2FsbGVkIGVhY2ggdGltZSBhIG5ldwo= --------------WhkBAh0eSaSrntIm9lE3EkTS--