Combining multiple Kokkos Tools using a common interface (PoC)

[ethan-puyaubreau]

This (draft for now) Pull Request presents a proof-of-concept that might enhance the flexibility of Kokkos Tools by enabling the simultaneous use of multiple profiling and debugging utilities at once. This PR introduces a combined tool template that would allow for multiple tools to function and communicate at once.

Key features of this proposal include:

• A unified interface for integrating and managing multiple Kokkos Tools within a single shared library.
• Comprehensive CMake configurations to streamline the build process for this combined approach.
• Illustrative example tools, CombinedSimple and CombinedDaemon, showcasing how this architecture allows profiling hooks to be effectively dispatched to different embedded tools. For instance, CombinedDaemon is an example of a tool evolving as a separate thread, capable of measuring data at regular time intervals while continuously listening for hook notifications. In this specific example, it merely logs timestamps, but it demonstrates the potential for regular data collection, a method that warrants further investigation.

This capability unlocks advanced analysis scenarios by allowing users to leverage a richer set of tools concurrently.

Note (outdated) : CombinedDaemon is not using jthread anymore, so it doesn't need C++20.

[ethan-puyaubreau]

Added commits in order to make my source files compliant with the clang format.

[masterleinad]

Note: To ensure the CombinedDaemon example functions correctly, a C++ compiler supporting std::jthread (introduced in C++17 afaik) is required for compilation. You might need to specify the C++17 standard in this case.

jthread is part of C++20.

[ethan-puyaubreau]

Thanks for the comment, I was not sure of this specification when I experimented with it. Rolled back my deamon example with a working thread system without C++20 jthread for now.

[dalg24]

Why are we pursuing composition at compile time as opposed as deciding at runtime however many tools to combine?

[JBludau]

Why are we pursuing composition at compile time as opposed as deciding at runtime however many tools to combine?

We can also do that, but then this tool will have a lot of dependencies ... we could ifdef them internally of course. Not sure what is the better approach here. I agree, that runtime decision is probably easier to understand for users

[vlkale]

This (draft for now) Pull Request presents a proof-of-concept that might enhance the flexibility of Kokkos Tools by enabling the simultaneous use of multiple profiling and debugging utilities at once. Currently, the KOKKOS_TOOLS_LIBS variable is restricted to loading only one shared library (.so). This PR introduces a combined tool template that would overcome this limitation.

Key features of this proposal include:

• A unified interface for integrating and managing multiple Kokkos Tools within a single shared library.
• Comprehensive CMake configurations to streamline the build process for this combined approach.
• Illustrative example tools, CombinedSimple and CombinedDaemon, showcasing how this architecture allows profiling hooks to be effectively dispatched to different embedded tools. For instance, CombinedDaemon is an example of a tool evolving as a separate thread, capable of measuring data at regular time intervals while continuously listening for hook notifications. In this specific example, it merely logs timestamps, but it demonstrates the potential for regular data collection, a method that warrants further investigation.

This capability unlocks advanced analysis scenarios by allowing users to leverage a richer set of tools concurrently.

Note (outdated) : CombinedDaemon is not using jthread anymore, so it doesn't need C++20.

This is a good direction and important. However, I would suggest looking at existing work on child libraries, particularly using the Kokkos Tools sampler as a reference. Additionally, are you aware of the ability to build a monolithic Kokkos Tools library via the Kokkos Tools CMakeLists.txt?

To elaborate:

First, looking closely at this PR, I don't know if the premise of current capabilities allowing for multiple tool libraries in KOKKOS_TOOLS_LIBS is quite true. One can definitely have at least two tools/utilities via child libraries. For example, it is possible to do:

export KOKKOS_TOOLS_LIBS="kp_sampler.so;kp_kernel_logger.so"

This works because the sampler has support for processing a child library; whether you can have two tools depends on whether the first library in the semi-colon separated list supports child libraries. See the code for the kokkos-sampler utility to see how support for child libraries is implemented.

Also, the second library can kp_nvtx_connector.so or any other library. Now, in the above example, if the kp_kernel_logger.so had support for child library, then one can put a third library in that list, i.e., one could do:

export  KOKKOS_TOOLS_LIBS="kp_sampler.so;kp_kernel_logger.so; kp_nvtx_connector.so"

Second, I want to check, are you aware that there is a KokkosTools_ENABLE_SINGLE option in CMakeLists.txt to build monolithic Kokkos Tools library that combines several specified tools into one .so file to pass to KOKKOS_TOOLS_LIBS?

[ethan-puyaubreau]

Indeed, I did see the ability to potentially have multiple tools within the same KOKKOS_TOOLS_LIBS setup. However, I wasn't aware of the ENABLE_SINGLE option that enables this combination mechanism. My main question is, would this allow for propagation and communication between say two tools that have been combined? I'll admit the interface I've created for this example/PR is quite primitive; my focus, for now, has primarily been on the Daemon part."

[vlkale]

Indeed, I did see the ability to potentially have multiple tools within the same KOKKOS_TOOLS_LIBS setup.

OK, good to know! Forgive me for boring you then with that explanation. I didn't see the discussion of this existing work of child libraries in this draft PR and didn't think your statement "Currently, the KOKKOS_TOOLS_LIBS variable is restricted to loading only one shared library (.so)" was quite accurate. So, I thought to make sure to clarify the goals and new functionality that this PR provides.

I think this may not be directly related to your PR or a priority, but you may want to look at another PR #117 on sampling overlays from @(DavidPoliakoff): #117 .

I closed it since it wasn't the highest priority at the time, but it certainly is of interest to me and take a look at it after you are done with this PR.

However, I wasn't aware of the ENABLE_SINGLE option that enables this combination mechanism.

Yes, and maybe you could reuse/integrate some of the existing CMakeLists.txt logic with your PR.

My main question is, would this allow for propagation and communication between say two tools that have been combined?

You are right - there is no support to communicate between two arbitrary tools (this had been on my wishlist/todos for some time). If your PR can communicate b/w 2 or more tools specified in a Kokkos Tools monolithic library, that would be great and would certainly add to new functionality to Kokkos Tools that does not exist.

I'll admit the interface I've created for this example/PR is quite primitive; my focus, for now, has primarily been on the Daemon part."

Yes, that is fine - it is good to start with a more scoped-down/concrete functionality and then generalize. Focusing on the Daemon part seems like an important use case as well.