common: verbose: asynchronous verbose mode for execution time tracking #3055
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
documentation
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
25b0638 to
bf1e8d1
Compare
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
bf1e8d1 to
625eec4
Compare
github-actions
bot
added
platform:gpu-intel
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
625eec4 to
e69f76d
Compare
github-actions
bot
removed
the
platform:gpu-generic
mgouicem
reviewed
May 12, 2025
mgouicem
reviewed
May 12, 2025
mgouicem
reviewed
May 12, 2025
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
2 times, most recently
from
c834a20 to
dc4f76d
Compare
avmanerikar
changed the title
echeresh
reviewed
May 27, 2025
src/gpu/intel/ocl/stream.cpp
Outdated
| return status::success; | ||
|
|
||
| } else { | ||
| cl_int err = clWaitForEvents(1, &async_tracked_event_); |
There was a problem hiding this comment.
Isn't this call synchronous? We enqueue a kernel, record an event and execution blocks here, until the kernel finishes. Am I missing something?
There was a problem hiding this comment.
Yes. This was a fallback for failure cases where the verbose info is then printed with the default stream.wait() calls. The implementation has been updated to avoid repetition.
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
dc4f76d to
8051a73
Compare
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
8051a73 to
ad7be6d
Compare
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
ad7be6d to
c9a7d45
Compare
github-actions
bot
removed
the
documentation
github-actions
bot
removed
component:api
avmanerikar
force-pushed
the
amanerik/main/async-verbose-mode
branch
from
c9a7d45 to
4513647
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR proposes a PoC for introducing an asynchronous verbose mode to accurately track kernel execution times in a non-blocking manner with minimal synchronization latencies. For the verbose mode, retrieving the kernel timing causes significant overhead as it requires the GPU kernel execution to be synchronized and also because it is tracked on the host.
The asynchronous mode removes the synchronization overhead by using event callbacks to query execution timings.
The prototype is created for a OpenCL GPU API that provides the kernel execution statistics for profiling.
The implementation enabled at run-time with
DNNL_ASYNC_VERBOSE=1:Related RFC: [link]
Addresses MFDNN-13603.
Checklist