Skip to content

Use caliper in distributed benchmarks#1274

Merged
aprokop merged 4 commits into
arborx:masterfrom
aprokop:add_caliper
May 16, 2026
Merged

Use caliper in distributed benchmarks#1274
aprokop merged 4 commits into
arborx:masterfrom
aprokop:add_caliper

Conversation

@aprokop

@aprokop aprokop commented Jun 21, 2025

Copy link
Copy Markdown
Contributor
export KOKKOS_TOOLS_LIBS="<path_to_caliper>/lib/libcaliper.so"
CALI_CONFIG="runtime-report(profile.mpi,profile.kokkos),calc.inclusive,max_column_width=80" mpirun <blah>

The only absolutely necessary thing from Caliper in ArborX is that we need to link to libcaliper on Mac. Otherwise, I'm getting

== CALIPER: (2): runtime-report: mpireport: MPI is already finalized. Cannot aggregate output.

See llnl/Caliper#737.

@aprokop aprokop requested review from Rombur and dalg24 June 21, 2025 13:11
Comment thread benchmarks/CMakeLists.txt Outdated
Comment thread benchmarks/distributed_tree_driver/distributed_tree_driver.cpp Outdated
Comment thread benchmarks/CMakeLists.txt
dalg24
dalg24 previously requested changes Jun 21, 2025
Comment thread benchmarks/distributed_tree_driver/distributed_tree_driver.cpp Outdated
Comment thread benchmarks/CMakeLists.txt
@aprokop

aprokop commented Jun 27, 2025

Copy link
Copy Markdown
Contributor Author

I simplified the Caliper code a bit. The specific options can be passed through CALI_CONFIG.

@aprokop aprokop requested a review from dalg24 June 30, 2025 19:07
@aprokop aprokop changed the title Use caliper in distributed benchmark Use caliper in distributed benchmarks Jul 1, 2025
@aprokop

aprokop commented May 13, 2026

Copy link
Copy Markdown
Contributor Author

I looked into this closer.

The only absolutely necessary thing from Caliper in ArborX is that we need to link to libcaliper. Otherwise, I'm getting

== CALIPER: (2): runtime-report: mpireport: MPI is already finalized. Cannot aggregate output.

and I haven't found a way to avoid that by using any of the environment options.

@aprokop

aprokop commented May 15, 2026

Copy link
Copy Markdown
Contributor Author

Caliper only requires additional linking on non-Linux systems. See llnl/Caliper#737.

Comment thread benchmarks/CMakeLists.txt Outdated

# In certain scenarios, using Caliper may require additional linking.
# See https://github.com/llnl/Caliper/issues/737
if(UNIX AND NOT APPLE)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not 100% certain this is the right way to indicate any Linux system. But I think it is close enough.

@aprokop aprokop requested a review from Rombur May 15, 2026 13:24
Comment thread benchmarks/CMakeLists.txt Outdated
# See https://github.com/llnl/Caliper/issues/737
if(UNIX AND NOT APPLE)
if(ARBORX_ENABLE_CALIPER)
message(WARNING "ARBORX_ENABLE_CALIPER was set to ON, but this is a non-Apple system. Forcing to OFF.")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This message is confusing if you don't know that there is no need to link Caliper on Linux. It reads as you cannot use Caliper on non-Apple system

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point. What would you suggest?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can rename the variable to sth like ARBORX_ENABLE_CALIPER_MACOS, since that's what it does. It's also clear that you shouldn't use it on linux

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any harm linking with Caliper so that yoiou have to disable it or error out?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no harm. The main point is that we don't want anyone to think they have to do that if they want to use Caliper on Linux.

Comment thread benchmarks/CMakeLists.txt Outdated
option(ARBORX_ENABLE_CALIPER "Enable Caliper instrumentation" OFF)
if(ARBORX_ENABLE_CALIPER)
option(ARBORX_ENABLE_CALIPER "Enable Caliper instrumentation" OFF)
if(ARBORX_ENABLE_CALIPER)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible for a user to set ARBORX_ENABLE_CALIPER=OFF and at the same time ARBORX_ENABLE_CALIPER_LINKING=ON or is ARBORX_ENABLE_CALIPER_LINKING only internal?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. Need to check that it does not happen.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the code so that ARBORX_ENABLE_CALIPER_LINKING is always set by us.

@aprokop aprokop merged commit b35c337 into arborx:master May 16, 2026
3 of 4 checks passed
@aprokop aprokop deleted the add_caliper branch May 16, 2026 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants