Skip to content

Various fixes for heterogeneous utilities #47605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion DataFormats/SoATemplate/interface/SoACommon.h
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,17 @@

#include <boost/preprocessor.hpp>

#ifdef __CUDACC__
#include <cuda_runtime.h>
#endif

#ifdef __HIPCC__
#include <hip/hip_runtime_api.h>
#endif

#include "FWCore/Utilities/interface/typedefs.h"

// CUDA attributes
// CUDA/ROCm attributes
#if defined(__CUDACC__) || defined(__HIPCC__)
#define SOA_HOST_ONLY __host__
#define SOA_DEVICE_ONLY __device__
Expand Down
1 change: 1 addition & 0 deletions DataFormats/SoATemplate/interface/SoALayout.h
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/

#include <cassert>
#include <cstring>

#include "FWCore/Reflection/interface/reflex.h"

Expand Down
2 changes: 1 addition & 1 deletion HeterogeneousCore/AlpakaCore/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The `BuildFile.xml` must contain `<flags ALPAKA_BACKENDS="1"/>` to enable the be
* If you need to transfer some data back to host, use `stream::SynchronizingEDProducer`
* All code using `ALPAKA_ACCELERATOR_NAMESPACE` should be placed in `Package/SubPackage/{interface,src,plugins,test}/alpaka` directory
* Alpaka-dependent code that uses templates instead of the namespace macro can be placed in `Package/SubPackage/interface` directory
* All source files (not headers) using Alpaka device code (such as kernel call, functions called by kernels) must have a suffic `.dev.cc`, and be placed in the aforementioned `alpaka` subdirectory
* All source files (not headers) using Alpaka device code (such as kernel call, functions called by kernels) must have a suffix `.dev.cc`, and be placed in the aforementioned `alpaka` subdirectory
* Any code that `#include`s a header from the framework or from the `HeterogeneousCore/AlpakaCore` must be separated from the Alpaka device code, and have the usual `.cc` suffix.
* Some framework headers are allowed to be used in `.dev.cc` files:
* Any header containing only macros, e.g. `FWCore/Utilities/interface/CMSUnrollLoop.h`, `FWCore/Utilities/interface/stringize.h`
Expand Down
15 changes: 15 additions & 0 deletions HeterogeneousCore/AlpakaInterface/interface/atomicInc.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#ifndef HeterogeneousCore_AlpakaInterface_interface_atomicInc_h
#define HeterogeneousCore_AlpakaInterface_interface_atomicInc_h

#include <alpaka/alpaka.hpp>

// This function is similar to atomicInc, but deduces the limiting value from the type itself.

ALPAKA_NO_HOST_ACC_WARNING
template <typename TAcc, typename T, typename THierarchy = alpaka::hierarchy::Grids>
ALPAKA_FN_HOST_ACC auto atomicInc(TAcc const& acc, T* address, THierarchy const& hierarchy = THierarchy()) -> T {
T limit = std::numeric_limits<T>::max();
return alpaka::atomicInc(acc, address, limit, hierarchy);
}

#endif // HeterogeneousCore_AlpakaInterface_interface_atomicInc_h
29 changes: 29 additions & 0 deletions HeterogeneousCore/AlpakaInterface/interface/atomicIncSaturate.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#ifndef HeterogeneousCore_AlpakaInterface_interface_atomicIncSaturate_h
#define HeterogeneousCore_AlpakaInterface_interface_atomicIncSaturate_h

#include <alpaka/alpaka.hpp>

// This function is similar to atomicInc, but instead of wrapping around it saturates at the given value.

ALPAKA_NO_HOST_ACC_WARNING
template <typename TAcc, typename T, typename THierarchy = alpaka::hierarchy::Grids>
ALPAKA_FN_HOST_ACC auto atomicIncSaturate(TAcc const& acc,
T* address,
T const& limit,
THierarchy const& hierarchy = THierarchy()) -> T {
T assumed;
T old = *address;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On host parallel backends (that we don't use at the moment) this line is not thread safe (actually, strictly speaking with the default Grids hierarchy level this line is not thread safe even with host serial backend if the same address is accessed concurrently from multiple kernels). To guarantee thread safety on host the load from address would have to be atomic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.
I will have to think how it can be implemented in a portable way, or specialised for the various backends.


do {
assumed = old;
if (assumed >= limit) {
// Saturate at limit.
break;
}
old = alpaka::atomicCas(acc, address, assumed, assumed + 1, hierarchy);
} while (old != assumed);

return old;
}

#endif // HeterogeneousCore_AlpakaInterface_interface_atomicIncSaturate_h