[Bug testsuite/104423] New: [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[Bug testsuite/104423] New: [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

            Bug ID: 104423
           Summary: [libgomp, testsuite] Add means to do accelerator-only
                    testing in libgomp
           Product: gcc
           Version: 12.0
            Status: UNCONFIRMED
          Severity: enhancement
          Priority: P3
         Component: testsuite
          Assignee: unassigned at gcc dot gnu.org
          Reporter: vries at gcc dot gnu.org
  Target Milestone: ---

For nvptx offload testing, I switched to an eGPU setup, and cycle through 4
boards (kepler, maxwell, pascal, turing) to do pre-commit testing.  Sometime in
the future that'll be 5, once I get an ampere board as well.

I run a script test-board.sh that iterates over various dimensions, which takes
about 5 hours per board.  In practice, this means testing a patch get spread
over 2 days.

Furthermore, I'm looking to extend the dimensions on which I test, possibly
doubling the test-board time.

I noticed that a lot of time is spent running libgomp tests that do not use the
accelerator, and which are not needed for my purposes.

It would be nice if libgomp provided a mode to run only tests that use the
accelerator, to drive down the test-board.sh execution time.

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

--- Comment #1 from Tom de Vries <vries at gcc dot gnu.org> ---
One of the dimensions that I test is env var GOMP_NVPTX_JIT, with values:
- -O0, and
- default (using unset GOMP_NVPTX_JIT), which supposedly is -O4.

Looking at f.i. test-case for-3.c, compilation takes 3 minutes and execution a
few seconds.  It would make sense to reuse the executable.

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[tschwinge at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

Thomas Schwinge <tschwinge at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tschwinge at gcc dot gnu.org
     Ever confirmed|0                           |1
           Keywords|                            |openacc, openmp
             Status|UNCONFIRMED                 |NEW
   Last reconfirmed|                            |2022-02-08
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=66005

--- Comment #2 from Thomas Schwinge <tschwinge at gcc dot gnu.org> ---
(In reply to Tom de Vries from comment #0)
> I noticed that a lot of time is spent running libgomp tests that do not use the accelerator, and which are not needed for my purposes.

We can assume that effectively all OpenACC test cases use any available GPUs.

For OpenMP test cases indeed there is no clear answer.

> It would be nice if libgomp provided a mode to run only tests that use the accelerator, to drive down the test-board.sh execution time.

We may add a knob to skip the OpenACC '-foffload=disable' testing.

For OpenMP test cases, we'd either have to manually mark them up (error prone
and generally ugly), or scan the source file(s) (error prone and generally
ugly), or use some kind of marker that gets set during compilation if any
offloading code has been generated.

Regarding the latter, you also reported:

(In reply to Tom de Vries from comment #1)
> Looking at f.i. test-case for-3.c, compilation takes 3 minutes

We should be able to at least mitigate that problem via my PR66005 idea.

> and execution a few seconds.  It would make sense to reuse the executable.

I suppose you mean re-use across what currently are independent libgomp
testsuite invocations:

(In reply to Tom de Vries from comment #1)
> One of the dimensions that I test is env var GOMP_NVPTX_JIT, with values:
> - -O0, and
> - default (using unset GOMP_NVPTX_JIT), which supposedly is -O4.

For OpenACC, we'd hook that into the existing looping over any available GPUs
(plus host fallback).

For OpenMP, there currently is no such looping.

If paralellizing compilation via PR66005 is not sufficient then indeed we have
to think about switching OpenACC testing to a mode where only once we do
compile (without any '-foffload=[...]), and then loop over 'ACC_DEVICE_TYPE'
etc.

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

--- Comment #3 from Tom de Vries <vries at gcc dot gnu.org> ---
(In reply to Thomas Schwinge from comment #2)
> For OpenMP test cases, we'd either have to manually mark them up (error
> prone and generally ugly), or scan the source file(s) (error prone and
> generally ugly), or use some kind of marker that gets set during compilation
> if any offloading code has been generated.

I was thinking along the lines of already existing markers in the exec:
...
$ nm for-3.exe | grep offload
                 U GOMP_offload_register_ver
                 U GOMP_offload_unregister_ver
0000000000a59718 D __offload_funcs_end
0000000000a58a58 d .offload_func_table
0000000000a58a58 D __offload_func_table
0000000000a5ca38 D __offload_vars_end
0000000000a59718 d .offload_var_table
0000000000a59718 D __offload_var_table
...

Say, if the exec doesn't have a __offload_func_table, it can be skipped.

This would allow to do one test run, where you run all the tests, and generate
the offloading tests list, and then several more runs (using the same sources,
so the list is still up to date) where you just do the accelerator tests.

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

--- Comment #4 from Tom de Vries <vries at gcc dot gnu.org> ---
Created attachment 52416
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52416&action=edit
[libgomp, testsuite, nvptx] Add GOMP_NVPTX_JIT_ITER (libgomp.c/c.exp only)

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[vries at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

--- Comment #5 from Tom de Vries <vries at gcc dot gnu.org> ---
Created attachment 52438
  --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=52438&action=edit
Tentative patch (GOMP_TARGET_ENV_ITER)

A more generic solution using env var GOMP_TARGET_ENV_ITER, which allows us to
specify: GOMP_TARGET_ENV_ITER="GOMP_NVPTX_JIT _ -O0".

[Bug testsuite/104423] [libgomp, testsuite] Add means to do accelerator-only testing in libgomp

[egallager at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104423

Eric Gallager <egallager at gcc dot gnu.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |egallager at gcc dot gnu.org
           See Also|                            |https://gcc.gnu.org/bugzill
                   |                            |a/show_bug.cgi?id=103324

--- Comment #6 from Eric Gallager <egallager at gcc dot gnu.org> ---
Seems kinda like the same idea as bug 103324, albeit a more-specific variant of
it just for libgomp.