* [Bug libgomp/113513] [OpenMP] libgomp: cuCtxGetDevice error with OMP_DISPLAY_ENV=true OMP_TARGET_OFFLOAD="mandatory" for libgomp.c/target-52.c
2024-01-19 23:25 [Bug libgomp/113513] New: [OpenMP] libgomp: cuCtxGetDevice error with OMP_DISPLAY_ENV=true OMP_TARGET_OFFLOAD="mandatory" for libgomp.c/target-52.c burnus at gcc dot gnu.org
@ 2024-01-20 14:38 ` burnus at gcc dot gnu.org
2024-01-22 19:45 ` burnus at gcc dot gnu.org
1 sibling, 0 replies; 3+ messages in thread
From: burnus at gcc dot gnu.org @ 2024-01-20 14:38 UTC (permalink / raw)
To: gcc-bugs
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=113513
--- Comment #1 from Tobias Burnus <burnus at gcc dot gnu.org> ---
Looking at the called GOMP_OFFLOAD_* function, in the failing case, there is:
...
DEBUG GOMP_OFFLOAD_run
DEBUG GOMP_OFFLOAD_dev2host
DEBUG GOMP_OFFLOAD_free
DEBUG: nvptx_attach_host_thread_to_device - 0
and in the successful case:
DEBUG GOMP_OFFLOAD_fini_device 0 <<< called before unregister
DEBUG: GOMP_offload_unregister_ver dev=0; state=GOMP_DEVICE_FINALIZED
DEBUG: GOMP_offload_unregister_ver dev=0; state=GOMP_DEVICE_FINALIZED
and then - in the failing case:
DEBUG: GOMP_offload_unregister_ver dev=0; state=GOMP_DEVICE_INITIALIZED
DEBUG: GOMP_offload_unregister_ver dev=0; state=GOMP_DEVICE_INITIALIZED
DEBUG: gomp_unload_image_from_device
DEBUG GOMP_OFFLOAD_unload_image, 0, 196609
DEBUG: gomp_target_fini; dev=0, state=GOMP_DEVICE_INITIALIZED
DEBUG GOMP_OFFLOAD_fini_device 0
DEBUG: nvptx_attach_host_thread_to_device - 0
libgomp: cuCtxGetDevice error: unknown cuda error
Thus, for some reason, GOMP_OFFLOAD_fini_device then
GOMP_offload_unregister_ver is swapped when
OMP_DISPLAY_ENV=true and OMP_TARGET_OFFLOAD="mandatory"
are set - but not otherwise.
The call to omp_target_fini comes from:
if (atexit (gomp_target_fini) != 0)
gomp_fatal ("atexit failed");
While the call to GOMP_offload_unregister_ver comes from mkoffload:
fprintf (out, "static __attribute__((destructor)) void fini (void)\n"
"{\n"
" GOMP_offload_unregister_ver (%#x, __OFFLOAD_TABLE__,"
" %d/*NVIDIA_PTX*/, &nvptx_data);\n"
"};\n",
* * *
Actually, the same problem occurs when compiled with:
-foffload=disable
With that flag + no 'mandatory':
DEBUG GOMP_OFFLOAD_version
DEBUG GOMP_OFFLOAD_get_caps
DEBUG GOMP_OFFLOAD_get_num_devices 0
DEBUG GOMP_OFFLOAD_get_name
DEBUG GOMP_OFFLOAD_get_type
DEBUG GOMP_OFFLOAD_init_device 0
DEBUG: nvptx_open_device - 0
DEBUG: gomp_target_fini; dev=0, state=GOMP_DEVICE_INITIALIZED
DEBUG GOMP_OFFLOAD_fini_device 0
DEBUG: nvptx_attach_host_thread_to_device - 0
And with 'mandatory' + OMP_DISPLAY_ENV=verbose:
DEBUG GOMP_OFFLOAD_version
DEBUG GOMP_OFFLOAD_get_caps
DEBUG GOMP_OFFLOAD_get_num_devices 0
DEBUG GOMP_OFFLOAD_get_name
DEBUG GOMP_OFFLOAD_get_type
< omp_display_env output>
DEBUG GOMP_OFFLOAD_init_device 0
DEBUG: nvptx_open_device - 0
libgomp: OMP_TARGET_OFFLOAD is set to MANDATORY, but device cannot be used for
offloading
DEBUG: gomp_target_fini; dev=0, state=GOMP_DEVICE_INITIALIZED
DEBUG GOMP_OFFLOAD_fini_device 0
DEBUG: nvptx_attach_host_thread_to_device -
libgomp: cuCtxGetDevice error: unknown cuda error
libgomp: device finalization failed
Thus, the error message is the same – but here no offloading code exists and
just gomp_target_fini is called. - However, there is a prior call to
'gomp_fatal' which probably messes things up for the plugin handling - while
in the original code, we have a valid code.
* * *
If there is no offloading code but
OMP_DISPLAY_ENV=verbose OMP_TARGET_OFFLOAD="mandatory"
is used, it works:
DEBUG GOMP_OFFLOAD_version
DEBUG GOMP_OFFLOAD_get_caps
DEBUG GOMP_OFFLOAD_get_num_devices 0
DEBUG GOMP_OFFLOAD_get_name
DEBUG GOMP_OFFLOAD_get_type
OPENMP DISPLAY ENVIRONMENT BEGIN
...
OPENMP DISPLAY ENVIRONMENT END
DEBUG: gomp_target_fini; dev=0, state=0
* * *
If there is only one or none of the two env vars, there is no need to search
for devices - and, hence, the nvptx plugin is not called at all and it,
obviously, works as well.
^ permalink raw reply [flat|nested] 3+ messages in thread