[Bug libgomp/101581] New: [OpenMP] omp_target_memcpy – support inter-device memcpy

[Bug libgomp/101581] New: [OpenMP] omp_target_memcpy – support inter-device memcpy

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101581

            Bug ID: 101581
           Summary: [OpenMP] omp_target_memcpy – support inter-device
                    memcpy
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Keywords: openmp
          Severity: normal
          Priority: P3
         Component: libgomp
          Assignee: unassigned at gcc dot gnu.org
          Reporter: burnus at gcc dot gnu.org
                CC: jakub at gcc dot gnu.org
  Target Milestone: ---

The current omp_target_memcpy implementation handles:

host/shared-mem device → host/shared-mem device
  if (src_devicep == NULL && dst_devicep == NULL)

host/shared-mem device → non-shared-mem device
  if (src_devicep == NULL)

non-shared-mem device → host/shared-mem device
  if (dst_devicep == NULL)

device → same device
  if (src_devicep == dst_devicep)

but it fails with
  return EINVAL;
when attempting to copy between two devices which don't have
GOMP_OFFLOAD_CAP_SHARED_MEM.

Expected: I do not see anything in the OpenMP spec which disallows copying
between non-shared-mem devices.  Thus, I think it should work. This probably
requires device -> host -> device copying with a temporary on the host.

Likewise for omp_target_memcpy_rect

[Bug libgomp/101581] [OpenMP] omp_target_memcpy – support inter-device memcpy

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101581

--- Comment #1 from Tobias Burnus <burnus at gcc dot gnu.org> ---
For omp_target_memcpy_rect, see:

"[patch] OpenMP: Call cuMemcpy2D/cuMemcpy3D for nvptx for
omp_target_memcpy_rect"
https://gcc.gnu.org/pipermail/gcc-patches/2023-July/625465.html

[Bug libgomp/101581] [OpenMP] omp_target_memcpy – support inter-device memcpy

[burnus at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101581

--- Comment #2 from Tobias Burnus <burnus at gcc dot gnu.org> ---
Thomas pointed out that, at least with CUDA, the cuMemcpyPeer function could be
used (and cuMemcpy3DPeer for PR110813).

(For cuMemcpy, there is also CU_MEMORYTYPE_UNIFIED which might be of some use.)

BTW: "[patch] OpenMP: Call cuMemcpy2D/cuMemcpy3D for nvptx for
omp_target_memcpy_rect"
has been committed as r14-2792-g25072a477a56a7