Re: [PATCH, nvptx, 1/2] Reimplement libgomp barriers for nvptx

[Chung-Lin Tang <chunglin.tang@gmail.com>]

On 2022/9/21 5:01 PM, Jakub Jelinek wrote:
> On Wed, Sep 21, 2022 at 03:45:36PM +0800, Chung-Lin Tang via Gcc-patches wrote:
>> Hi Tom,
>> I had a patch submitted earlier, where I reported that the current way of implementing
>> barriers in libgomp on nvptx created a quite significant performance drop on some SPEChpc2021
>> benchmarks:
>> https://gcc.gnu.org/pipermail/gcc-patches/2022-September/600818.html
>>
>> That previous patch wasn't accepted well (admittedly, it was kind of a hack).
>> So in this patch, I tried to (mostly) re-implement team-barriers for NVPTX.
>>
>> Basically, instead of trying to have the GPU do CPU-with-OS-like things that it isn't suited for,
>> barriers are implemented simplistically with bar.* synchronization instructions.
>> Tasks are processed after threads have joined, and only if team->task_count != 0
>>
>> (arguably, there might be a little bit of performance forfeited where earlier arriving threads
>> could've been used to process tasks ahead of other threads. But that again falls into requiring
>> implementing complex futex-wait/wake like behavior. Really, that kind of tasking is not what target
>> offloading is usually used for)
> 
> I admit I don't have a good picture if people in real-world actually use
> tasking in offloading regions and how much and in what way, but the above
> definitely would be a show-stopper for typical tasking workloads, where
> one thread (usually from master/masked/single construct's body) creates lots
> of tasks and can spend considerable amount of time in those preparations,
> while other threads are expected to handle those tasks.

I think the most common use case for target offloading is "parallel for".

Really, not simply removing tasking altogether from target regions in the specification is just looking for trouble.

If asynchronous offloaded tasks are to be supported, something at the whole GPU offload region level
is much more reasonable, like the async clause functionality in OpenACC.

> Do we have an idea how are other implementations handling this?
> I think it should be easily observable with atomics, have
> master/masked/single that creates lots of tasks and then spends a long time
> doing something, have very small task bodies that just increment some atomic
> counter and at the end of the master/masked/single see how many tasks were
> already encountered.

This could be an interesting test...

> Note, I don't have any smart ideas how to handle this instead and what
> you posted might be ok for what people usually do on offloading targets
> in OpenMP if they use tasking at all, just wanted to mention that there
> could be workloads where the above is a serious problem.  If there are
> say hundreds of threads doing nothing until a single thread reaches a
> barrier and there are hundreds of pending tasks...

I think it might still be doable, just not in the very fine "wake one thread" style
that the Linux-based implementation was doing.

> E.g. note we have that 64 pending task limit after which we start to
> create undeferred tasks, so if we never start handling tasks until
> one thread is done with them, that would mean the single thread
> would create 64 deferred tasks and then handle all the others itself
> making it even longer until the other tasks can deal with it.

Okay, thanks for reminding that.

Chung-Lin