Clarification on synchronization semantics for reductions on HIP vs CUDA backends

[gzagaris]

Describe the bug

I encountered a situation where the synchronization behavior of loops with reductions appears to be different between HIP and CUDA backends.

Consider the following code snippet:

template < typename loop_exec, typename reduce_exec, typename T, typename SizeT >
T raja_sum_reduce(T* data, SizeT N) noexcept
{
  RAJA::ReduceSum< reduce_exec, T > sum(T(0));

  RAJA::forall< loop_exec >(
    RAJA::RangeSegment(SizeT(0), N), RAJA_DEVICE(SizeT i) {
      sum += data[i];
  });

  // For HIP it seems that I have to explicitly synchronize before calling sum.get()
  T val = static_cast<T>(sum.get());
  return val;
}

For HIP, I am calling this with:

raja_sum_reduce< RAJA::hip_exec< 256 >, RAJA::hip_reduce >(data, N);

And for CUDA, I am calling this with:

raja_sum_reduce< RAJA::cuda_exec< 256 >, RAJA::cuda_reduce >(data, N);

Observed Behavior

With the HIP backend, it seems that I have to call RAJA::synchronize() before calling sum.get(). However, with the CUDA backend everything works as expected.

I encountered this in the context of a larger application, but, I haven't been able to reproduce it with a minimal standalone RAJA test.

Questions

1. Does the call to sum.get() guarantee synchronization or should the application explicitly synchronize even though the RAJA::forall execution policy is not asynchronous (async)?

2. Is the behavior of reductions identical between the CUDA and HIP backends?

3. I noticed that there are some additional execution policies for loops that have reductions, RAJA::cuda_exec_with_reduce< BLOCK_SIZE > and RAJA::hip_exec_with_reduce< BLOCK_SIZE > respectively.

   • Is the current guidance to use these policies with RAJA::forall instead?
   • How are these policies different from the normal RAJA::cuda_exec< BLOCK_SIZE > and RAJA::hip_exec< BLOCK_SIZE > execution policies?

Thank you very much for all your guidance and help.

[MrBurmark]

Guidance on policy usage, as much as exists, is in cookbook sections of the RAJA User's guide. https://raja.readthedocs.io/en/develop/sphinx/user_guide/cook_book/reduction.html
The policies may differ in how they map work to threads. Generally policies optimized for reduce will do something like a grid stride loop so that the minimal number of blocks are launched and hence the minimal size reduction tree is required to get the final answer. This means that each thread loops over a runtime number of iterations roughly equal to the size of the iteration space divided by the number of threads in the kernel at maximum occupancy as given by the cuda/hip occupancy calculator functions.

[MrBurmark]

The get call of a reducer synchronizes enough so that the result may be obtained. What this means specifically for the current reducer implementation is that wait is called on the set of camp resources used in the kernels where the reducer was used/captured. So each cuda/hip stream that the reducer was used with is synchronized. Note that the default cuda/hip resource does not correspond with the cuda/hip default stream (unless camp was configured that way), so work in other streams will not be synchronized. The cuda and hip backends are essentially identical.

[gzagaris]

Note that the default cuda/hip resource does not correspond with the cuda/hip default stream (unless camp was configured that way), so work in other streams will not be synchronized. The cuda and hip backends are essentially identical.

Thank you -- this is very helpful. I think this statement above likely explains what might be happening. If I understand correctly your explanation, the reducer synchronizes the streams that are associated with the reducer's camp resources, but, does not synchronize work on other stream (e.g., the default cuda/hipstream). This was likely being masked with CUDA due to stream ordering behavior (?), but, was exposed with the HIP backend.

[MrBurmark]

Another thought, neither the kernel nor the reducer will synchronize if the iteration space is empty.

[gzagaris]

Another thought, neither the kernel nor the reducer will synchronize if the iteration space is empty.

That's good to know. However, in this case, the iteration space is definitely non-empty, so I don't think I am running into this issue. I added explicit synchronization in my wrappers, which resolves the issue. Thanks again for your help!