Support export_model and import_model for MULTI device mode#33898
Closed
andersendsa wants to merge 6 commits intoopenvinotoolkit:masterfrom
Closed
Support export_model and import_model for MULTI device mode#33898andersendsa wants to merge 6 commits intoopenvinotoolkit:masterfrom
andersendsa wants to merge 6 commits intoopenvinotoolkit:masterfrom
Conversation
Motivation: Exporting a compiled model in MULTI mode was previously not supported, returning OPENVINO_NOT_IMPLEMENTED. This prevented users from caching or serializing MULTI-device models. Changes: - Implemented `AutoCumuCompiledModel::export_model` in `src/plugins/auto/src/cumulative_compiled_model.cpp` to serialize the MULTI configuration (XML) and delegate model export to sub-devices. - Implemented `Plugin::import_model` in `src/plugins/auto/src/plugin.cpp` to deserialize the MULTI configuration and reconstruct the distributed model state by importing sub-models. - Updated `CumuSchedule::init` in `src/plugins/auto/src/cumulative_schedule.cpp` to handle initialization during import (skipping compilation tasks when `ov::Model` is null). - Added `openvino/pass/serialize.hpp` and `openvino/util/xml_parse_utils.hpp` includes where necessary. - Added a functional test `CanExportImportMultiModel` in `src/plugins/auto/tests/functional/behavior/multi_export_import_test.cpp`. Verification: - Compiled `openvino_auto_plugin` and `ov_auto_func_tests`. - Ran `AutoFuncTests.CanExportImportMultiModel` test, which passed successfully. - Verified that the exported model can be imported and used for inference with correct results and device utilization. Co-authored-by: andersendsa <199610634+andersendsa@users.noreply.github.com>
github-actions
bot
added
the
category: AUTO
Author
|
hello @praasz , @t-jankowski , @alvoron this is the pr i have made let me know if there any changes to be made or is it good to merge |
Motivation: Exporting a compiled model in MULTI mode was previously not supported, returning OPENVINO_NOT_IMPLEMENTED. This prevented users from caching or serializing MULTI-device models. Changes: - Implemented `AutoCumuCompiledModel::export_model` in `src/plugins/auto/src/cumulative_compiled_model.cpp` to serialize the MULTI configuration (XML) and delegate model export to sub-devices. - Implemented `Plugin::import_model` in `src/plugins/auto/src/plugin.cpp` to deserialize the MULTI configuration and reconstruct the distributed model state by importing sub-models. - Updated `CumuSchedule::init` in `src/plugins/auto/src/cumulative_schedule.cpp` to handle initialization during import (skipping compilation tasks when `ov::Model` is null). - Added `openvino/pass/serialize.hpp` and `openvino/util/xml_parse_utils.hpp` includes where necessary. - Added a functional test `CanExportImportMultiModel` in `src/plugins/auto/tests/functional/behavior/multi_export_import_test.cpp`. - Updated copyright years to 2026. Verification: - Compiled `openvino_auto_plugin` and `ov_auto_func_tests`. - Ran `AutoFuncTests.CanExportImportMultiModel` test, which passed successfully. - Verified that the exported model can be imported and used for inference with correct results and device utilization. Co-authored-by: andersendsa <199610634+andersendsa@users.noreply.github.com>
andersendsa
requested review from
artanokhov,
p-wysocki and
tsavina
and removed request for
a team
github-actions
bot
added
category: inference
Motivation: Enable `export_model` and `import_model` functionality for the MULTI device plugin to allow saving and loading compiled models for Cumulative Throughput mode. Also fix a critical CI failure in `Smart_CI` action. Changes: - Implemented `AutoCumuCompiledModel::export_model` to serialize device configurations to XML and sub-device binaries to the stream. - Implemented `Plugin::import_model` to parse the XML and reconstruct the distributed compiled model. - Updated `CumuSchedule` initialization to support loading without an initial `ov::Model`. - Added functional test `CanExportImportMultiModel` covering export/import flow. - Fixed race condition in `export_model` by capturing device state under lock. - Removed faulty `StreamSerialize` fallback. - Updated `.github/actions/smart-ci/action.yml` to use isolated paths for internal checkouts, preventing workspace cleanup issues. Verification: - Added `AutoFuncTests.CanExportImportMultiModel` passes. - Verified fix for race condition and XML formatting. - CI fix addresses `requirements.txt` not found error. Co-authored-by: andersendsa <199610634+andersendsa@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
additions made :
adding export and import functionality to multi-mode
closes issue #33889
hey @mengyays kindly review the pr