Re: [PATCH, OpenACC, v3] Non-contiguous array support for OpenACC data clauses

[Thomas Schwinge <thomas@codesourcery.com>]

[-- Attachment #1: Type: text/plain, Size: 2915 bytes --]

Hi!

On 2023-03-15T15:47:47+0100, I wrote:
> On 2019-11-26T22:49:21+0800, Chung-Lin Tang <chunglin_tang@mentor.com> wrote:
>> this is a reorg of the last non-contiguous arrays patch.
>
> (Sorry, this is still not the master branch integration email...)
>
>
> I noticed the following while working on something else:
>
>> --- libgomp/oacc-parallel.c   (revision 278656)
>> +++ libgomp/oacc-parallel.c   (working copy)
>
>> @@ -311,8 +488,10 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (voi
>>
>>    goacc_aq aq = get_goacc_asyncqueue (async);
>>
>> -  tgt = gomp_map_vars_async (acc_dev, aq, mapnum, hostaddrs, NULL, sizes, kinds,
>> -                          true, GOMP_MAP_VARS_OPENACC);
>> +  tgt = gomp_map_vars_openacc (acc_dev, aq, mapnum, hostaddrs, sizes, kinds,
>> +                            nca_info);
>> +  free (nca_info);
>
> Given OpenACC 'async', don't we have to defer 'free' of the
> non-contiguous array support data structure here?  But I'm not completely
> sure -- can we rule out that any asynchronous copying of any data of
> 'nca_info' is still going on after returning from 'gomp_map_vars'?
>
>> @@ -488,9 +666,19 @@ GOACC_data_start (int flags_m, size_t mapnum,
>
>> -  tgt = gomp_map_vars (acc_dev, mapnum, hostaddrs, NULL, sizes, kinds, true,
>> -                    GOMP_MAP_VARS_OPENACC);
>> +  tgt = gomp_map_vars_openacc (acc_dev, NULL, mapnum, hostaddrs, sizes, kinds,
>> +                            nca_info);
>> +  free (nca_info);
>
> Here, it's not relevant, as there is no 'async' support (yet) for 'data'
> constructs.
>
>> --- libgomp/target.c  (revision 278656)
>> +++ libgomp/target.c  (working copy)
>
>> @@ -1044,6 +1114,98 @@ gomp_map_vars_internal (struct gomp_device_descr *
>
>> +               void *ptrblock = goacc_noncontig_array_create_ptrblock
>> +                 (nca, target_ptrblock);
>> +               gomp_copy_host2dev (devicep, aq, target_ptrblock, ptrblock,
>> +                                   nca->ptrblock_size, cbufp);
>> +               free (ptrblock);
>
> Here again, however, don't we have to defer the 'free'?
>
> Please verify the attached
> "Given OpenACC 'async', defer 'free' of non-contiguous array support data structures",
> in particular the 'libgomp/oacc-parallel.c:GOACC_parallel_keyed' case.

To allow me to make progress with the "something else" (depending on
this), I've now pushed to devel/omp/gcc-12
commit a1f6758ae08fa748b291954371859e0158d4d667
"Given OpenACC 'async', defer 'free' of non-contiguous array support data structures [PR76739]",
see attached.


Grüße
 Thomas


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Given-OpenACC-async-defer-free-of-non-contiguous-arr.patch --]
[-- Type: text/x-diff, Size: 2403 bytes --]

From a1f6758ae08fa748b291954371859e0158d4d667 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Wed, 15 Mar 2023 13:34:02 +0100
Subject: [PATCH] Given OpenACC 'async', defer 'free' of non-contiguous array
 support data structures [PR76739]

Fix-up for og12 commit 15d0f61a7fecdc8fd12857c40879ea3730f6d99f
"Merge non-contiguous array support patches".

	PR other/76739
	libgomp/
	* oacc-parallel.c (GOACC_parallel_keyed): Given OpenACC 'async',
	defer 'free' of non-contiguous array support data structures.
	* target.c (gomp_map_vars_internal): Likewise.
---
 libgomp/ChangeLog.omp   | 7 +++++++
 libgomp/oacc-parallel.c | 5 ++++-
 libgomp/target.c        | 6 +++++-
 3 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/libgomp/ChangeLog.omp b/libgomp/ChangeLog.omp
index ace54f2f82f..85ebab14ba8 100644
--- a/libgomp/ChangeLog.omp
+++ b/libgomp/ChangeLog.omp
@@ -1,3 +1,10 @@
+2023-03-24  Thomas Schwinge  <thomas@codesourcery.com>
+
+	PR other/76739
+	* oacc-parallel.c (GOACC_parallel_keyed): Given OpenACC 'async',
+	defer 'free' of non-contiguous array support data structures.
+	* target.c (gomp_map_vars_internal): Likewise.
+
 2023-03-23  Tobias Burnus  <tobias@codesourcery.com>
 
 	* testsuite/libgomp.fortran/map-alloc-comp-8.f90: New test.
diff --git a/libgomp/oacc-parallel.c b/libgomp/oacc-parallel.c
index 9cd99b4a0b4..136702d6e61 100644
--- a/libgomp/oacc-parallel.c
+++ b/libgomp/oacc-parallel.c
@@ -470,7 +470,10 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (void *),
     = goacc_map_vars (acc_dev, aq, mapnum, hostaddrs, NULL, sizes, kinds,
 		      nca_info, true, GOMP_MAP_VARS_TARGET);
 
-  free (nca_info);
+  if (aq == NULL)
+    free (nca_info);
+  else
+    acc_dev->openacc.async.queue_callback_func (aq, free, nca_info);
 
   if (profiling_p)
     {
diff --git a/libgomp/target.c b/libgomp/target.c
index 96ece0b31fd..aaa597f6610 100644
--- a/libgomp/target.c
+++ b/libgomp/target.c
@@ -1938,7 +1938,11 @@ gomp_map_vars_internal (struct gomp_device_descr *devicep,
 		    (nca, target_ptrblock);
 		  gomp_copy_host2dev (devicep, aq, target_ptrblock, ptrblock,
 				      nca->ptrblock_size, false, cbufp);
-		  free (ptrblock);
+		  if (aq)
+		    /* Free once the transfer has completed.  */
+		    devicep->openacc.async.queue_callback_func (aq, free, ptrblock);
+		  else
+		    free (ptrblock);
 		}
 	    }
 	}
-- 
2.25.1