[NV EP] fix EP context options #24545
Conversation
|
looks good. Thanks, Max, for the fix and additional cleanup! |
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
please address lintrunner failure |
|
Since we haven't set up a pipeline for NV EP, i tested it locally and encountered following compile error. ... D:\ort\onnxruntime\core\providers\nv_tensorrt_rtx\nv_provider_factory_creator.h(16,80): error C2143: syntax error: missing ',' before '*' [D:\ort\build\Windows\Debug\onnxruntime_session.vcxproj] D:\ort\onnxruntime\core\session\provider_registration.cc(292,61): error C2665: 'onnxruntime::NvProviderFactoryCreator::Create': no overloaded function could convert all the argument types [D:\ort\build\Window |
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
/azp run Test Linux CUDA x64 Release, Test Linux TensorRT x64 Release, web_Debug / build_onnxruntime_web, web_Release / build_onnxruntime_web |
|
No pipelines are associated with this pull request. |
|
/azp run Windows ARM64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
@anujj noticed that during rebasing I accidentally deleted the profile shape parsing. Just reverted that. |
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider.cc
Outdated
Show resolved
Hide resolved
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
|
the 2 conflicts are due to #24552 |
gedoensmax
force-pushed
the
fix_ep_context_nv
branch
from
dfbc6d3 to
ffa63b0
Compare
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline, Windows x64 QNN CI Pipeline |
|
Azure Pipelines successfully started running 5 pipeline(s). |
### Description While cleaning up the options I missed the part in the provider bridge that translates session options to TRT options. To better integrate with current IHV work I adopted the principle that QNN and OV use to pipe through session options. Since all this is string based magic it would be great to be access a general point of truth like `EpContextModelGenerationOptions` in the provider wrappedtypes. https://github.com/microsoft/onnxruntime/blob/6df620675290d97d7e406faf232b8b521333b6e8/onnxruntime/core/framework/session_options.h#L73 This is a fix on top of #24456 @ankan-ban and @chilo-ms to review.
### Description Cherry pick the following into [rel-1.22.0](https://github.com/microsoft/onnxruntime/tree/rel-1.22.0) - (#24608) - (#24545) --------- Co-authored-by: Changming Sun <[email protected]> Co-authored-by: Maximilian Müller <[email protected]>
Description
While cleaning up the options I missed the part in the provider bridge that translates session options to TRT options.
To better integrate with current IHV work I adopted the principle that QNN and OV use to pipe through session options. Since all this is string based magic it would be great to be access a general point of truth like
EpContextModelGenerationOptionsin the provider wrappedtypes.onnxruntime/onnxruntime/core/framework/session_options.h
Line 73 in 6df6206
This is a fix on top of #24456 @ankan-ban and @chilo-ms to review.