[Bug libgomp/97384] New: [libgomp, nvptx] Handle -msoft-stack-reserve-local=<n> overflow in plugin

[Bug libgomp/97384] New: [libgomp, nvptx] Handle -msoft-stack-reserve-local=<n> overflow in plugin

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97384

            Bug ID: 97384
           Summary: [libgomp, nvptx] Handle -msoft-stack-reserve-local=<n>
                    overflow in plugin
           Product: gcc
           Version: 11.0
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P3
         Component: libgomp
          Assignee: unassigned at gcc dot gnu.org
          Reporter: vries at gcc dot gnu.org
                CC: jakub at gcc dot gnu.org
  Target Milestone: ---

Using the option -msoft-stack-reserve-local=<n> results in a:
...
.local .align 8 .b8 %simtstack_ar[n+8];
...

However, the CU_LIMIT_STACK_SIZE is set by default to 1kb for my card/driver
combo, so if I specify say -msoft-stack-reserve-local=2048, I run into:
...
libgomp: cuCtxSynchronize error: an illegal memory access was encountered
...
or:
...
libgomp: cuCtxSynchronize error: an illegal instruction was encountered
...
[ The latter at GOMP_NVPTX_JIT=-O0. ] Which may look a lot like the behaviour
we're trying to fix by adding -msoft-stack-reserve-local.

There's currently no way to make this work.

We could add an env var, say GOMP_NVPTX_LIMIT_STACK_SIZE which is used to set:
...
  r = cuCtxSetLimit(CU_LIMIT_STACK_SIZE, gomp_nvptx_limit_stack_size);
...
and then do:
...
$ GOMP_NVPTX_LIMIT_STACK_SIZE=3072 ./a.out
...
[ Note that GOMP_NVPTX_LIMIT_STACK_SIZE id chosen to be larger than 2048 to
accommodate for other .local usage. ]

[ It would be nice if we could attempt to accommodate the requested stack size
in the libgomp plugin automatically.  In the current setup, that would mean
scanning the ptx code for "simtstack_ar[<n>]", which is a bit cumbersome and
probably too slow.  Perhaps emitting an additional additional line before the
pre-amble like this:
...
// SIMTSTACK_AR_SIZE: 2048
...
would be possible to handle quick enough. ]

[Bug libgomp/97384] [libgomp, nvptx] Handle -msoft-stack-reserve-local=<n> overflow in plugin

[cvs-commit at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97384

--- Comment #1 from GCC Commits <cvs-commit at gcc dot gnu.org> ---
The trunk branch has been updated by Thomas Schwinge <tschwinge@gcc.gnu.org>:

https://gcc.gnu.org/g:0d25989d60d15866ef4737d66e02432f50717255

commit r15-1069-g0d25989d60d15866ef4737d66e02432f50717255
Author: Thomas Schwinge <tschwinge@baylibre.com>
Date:   Fri May 31 17:04:39 2024 +0200

    nvptx offloading: 'GOMP_NVPTX_NATIVE_GPU_THREAD_STACK_SIZE' environment
variable [PR97384, PR105274]

    ... as a means to manually set the "native" GPU thread stack size.

            PR libgomp/97384
            PR libgomp/105274
            libgomp/
            * plugin/cuda-lib.def (cuCtxSetLimit): Add.
            * plugin/plugin-nvptx.c (nvptx_open_device): Handle
            'GOMP_NVPTX_NATIVE_GPU_THREAD_STACK_SIZE' environment variable.