Re: [PATCH, OpenACC, v3] Non-contiguous array support for OpenACC data clauses

[Thomas Schwinge <thomas@codesourcery.com>]

[-- Attachment #1: Type: text/plain, Size: 2542 bytes --]

Hi Chung-Lin!

On 2019-11-26T22:49:21+0800, Chung-Lin Tang <chunglin_tang@mentor.com> wrote:
> this is a reorg of the last non-contiguous arrays patch.

(Sorry, this is still not the master branch integration email...)


I noticed the following while working on something else:

> --- libgomp/oacc-parallel.c   (revision 278656)
> +++ libgomp/oacc-parallel.c   (working copy)

> @@ -311,8 +488,10 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (voi
>
>    goacc_aq aq = get_goacc_asyncqueue (async);
>
> -  tgt = gomp_map_vars_async (acc_dev, aq, mapnum, hostaddrs, NULL, sizes, kinds,
> -                          true, GOMP_MAP_VARS_OPENACC);
> +  tgt = gomp_map_vars_openacc (acc_dev, aq, mapnum, hostaddrs, sizes, kinds,
> +                            nca_info);
> +  free (nca_info);

Given OpenACC 'async', don't we have to defer 'free' of the
non-contiguous array support data structure here?  But I'm not completely
sure -- can we rule out that any asynchronous copying of any data of
'nca_info' is still going on after returning from 'gomp_map_vars'?

> @@ -488,9 +666,19 @@ GOACC_data_start (int flags_m, size_t mapnum,

> -  tgt = gomp_map_vars (acc_dev, mapnum, hostaddrs, NULL, sizes, kinds, true,
> -                    GOMP_MAP_VARS_OPENACC);
> +  tgt = gomp_map_vars_openacc (acc_dev, NULL, mapnum, hostaddrs, sizes, kinds,
> +                            nca_info);
> +  free (nca_info);

Here, it's not relevant, as there is no 'async' support (yet) for 'data'
constructs.

> --- libgomp/target.c  (revision 278656)
> +++ libgomp/target.c  (working copy)

> @@ -1044,6 +1114,98 @@ gomp_map_vars_internal (struct gomp_device_descr *

> +               void *ptrblock = goacc_noncontig_array_create_ptrblock
> +                 (nca, target_ptrblock);
> +               gomp_copy_host2dev (devicep, aq, target_ptrblock, ptrblock,
> +                                   nca->ptrblock_size, cbufp);
> +               free (ptrblock);

Here again, however, don't we have to defer the 'free'?

Please verify the attached
"Given OpenACC 'async', defer 'free' of non-contiguous array support data structures",
in particular the 'libgomp/oacc-parallel.c:GOACC_parallel_keyed' case.


Grüße
 Thomas


-----------------
Siemens Electronic Design Automation GmbH; Anschrift: Arnulfstraße 201, 80634 München; Gesellschaft mit beschränkter Haftung; Geschäftsführer: Thomas Heurung, Frank Thürauf; Sitz der Gesellschaft: München; Registergericht München, HRB 106955

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Given-OpenACC-async-defer-free-of-non-contiguous-arr.patch --]
[-- Type: text/x-diff, Size: 1794 bytes --]

From 998f1156a51010490c2e918defb4517706916c92 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@codesourcery.com>
Date: Wed, 15 Mar 2023 13:34:02 +0100
Subject: [PATCH] Given OpenACC 'async', defer 'free' of non-contiguous array
 support data structures

Fix-up for og12 commit 15d0f61a7fecdc8fd12857c40879ea3730f6d99f
"Merge non-contiguous array support patches".

	libgomp/
	* oacc-parallel.c (GOACC_parallel_keyed): Given OpenACC 'async',
	defer 'free' of non-contiguous array support data structures.
	* target.c (gomp_map_vars_internal): Likewise.
---
 libgomp/oacc-parallel.c | 5 ++++-
 libgomp/target.c        | 6 +++++-
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/libgomp/oacc-parallel.c b/libgomp/oacc-parallel.c
index 9cd99b4a0b4..136702d6e61 100644
--- a/libgomp/oacc-parallel.c
+++ b/libgomp/oacc-parallel.c
@@ -470,7 +470,10 @@ GOACC_parallel_keyed (int flags_m, void (*fn) (void *),
     = goacc_map_vars (acc_dev, aq, mapnum, hostaddrs, NULL, sizes, kinds,
 		      nca_info, true, GOMP_MAP_VARS_TARGET);
 
-  free (nca_info);
+  if (aq == NULL)
+    free (nca_info);
+  else
+    acc_dev->openacc.async.queue_callback_func (aq, free, nca_info);
 
   if (profiling_p)
     {
diff --git a/libgomp/target.c b/libgomp/target.c
index 96ece0b31fd..aaa597f6610 100644
--- a/libgomp/target.c
+++ b/libgomp/target.c
@@ -1938,7 +1938,11 @@ gomp_map_vars_internal (struct gomp_device_descr *devicep,
 		    (nca, target_ptrblock);
 		  gomp_copy_host2dev (devicep, aq, target_ptrblock, ptrblock,
 				      nca->ptrblock_size, false, cbufp);
-		  free (ptrblock);
+		  if (aq)
+		    /* Free once the transfer has completed.  */
+		    devicep->openacc.async.queue_callback_func (aq, free, ptrblock);
+		  else
+		    free (ptrblock);
 		}
 	    }
 	}
-- 
2.25.1