Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add trace event control for ORT Web performance profiling #23393

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

qwu16
Copy link

@qwu16 qwu16 commented Jan 16, 2025

Description

Add trace event control to better profile ORT web performance

Motivation and Context

ORT Web's current tracing implementation lacks interfaces for performance profiling using about://tracing. This PR introduces these interfaces, enabling performance bottleneck identification in ORT Web and adding several trace events for WebNN.

@qwu16
Copy link
Author

qwu16 commented Jan 16, 2025

@microsoft-github-policy-service agree company="Intel"

@qwu16
Copy link
Author

qwu16 commented Jan 16, 2025

@Honry Please help review this PR, thanks~

Copy link
Contributor

@Honry Honry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fs-eire, please help review the type script files. Thanks!

@@ -37,6 +42,11 @@ common::Status DataTransfer::CopyTensor(const Tensor& src, Tensor& dst) const {
}
}

if (trace) {
console.call<void>("timeEnd", emscripten::val("ORT::DataTransfer::CopyTensor"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to distinguish the tensor downloading and tensor uploading? If so, you can add event tracing to both if and else branch.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tensor downloading and uploading can be distinguished through MLContext::writeTensor and MLContenxt::readTensor event in tracing, so i simply add this trace event for both tensor downloading and uploading.

@@ -178,6 +183,9 @@ onnxruntime::common::Status Model::Dispatch(const InlinedHashMap<std::string, On
promises.call<void>("push", ml_tensor);
}
auto ml_tensors = emscripten::val::global("Promise").call<emscripten::val>("all", promises).await();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd suggest just adding the console.time before this line, Promise.all is the major time consumer for constructing the ml tensors.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can get a high level view of time spending before wasm Dispatch API call if Promise.all is included.

@guschmue guschmue added the ep:WebNN WebNN execution provider label Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ep:WebNN WebNN execution provider
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants