Skip to content

Commit 039d681

Browse files
authored
Make the separate communication stream optional (elemental#184)
If this option is not set, the communication will land on the Hydrogen-default stream. This _could_ be a separate stream from the data compute stream, but in the LBANN use case, it generally will be the compute stream.
1 parent e6aea8d commit 039d681

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

include/El/core/imports/aluminum.hpp

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,10 +359,25 @@ struct SyncInfoManager<Device::GPU>
359359
};
360360
#endif // HYDROGEN_HAVE_GPU
361361

362+
inline bool use_separate_comm_stream() noexcept
363+
{
364+
char const* const env = std::getenv("H_USE_SEPARATE_COMM_STREAM");
365+
return (env && std::strlen(env) && env[0] != '0');
366+
}
367+
362368
template <typename BackendT>
363369
SyncInfo<DeviceForBackend<BackendT>()> const& BackendSyncInfo()
364370
{
365371
constexpr Device D = DeviceForBackend<BackendT>();
372+
if constexpr (D == El::Device::GPU)
373+
{
374+
static bool const use_separate_stream = use_separate_comm_stream();
375+
if (!use_separate_stream)
376+
{
377+
return El::gpu::DefaultSyncInfo();
378+
}
379+
}
380+
366381
static SyncInfoManager<D> si_mgr_(BackendT::Name());
367382
return si_mgr_.si_;
368383
}

0 commit comments

Comments
 (0)