[NPUW][VLM][AUTO] Using Auto plugin for VLMs embeddings when running on NPU

[AlexanderKalistratov]

Description

• Using AUTO:GPU,CPU for embedding models instead of CPU.
• Reduces TTFT for 60-80%.
• NPU plugin doesn't support RemoteTensor so avoid passing it
• Updating AUTO device properties with default values instead of hardcoding it. It makes device properties configurable by user

EISW-195191

Checklist:

• Tests have been updated or added to cover the new code.
• This patch fully addresses the ticket.
• I have made corresponding changes to the documentation.

[Copilot]

Pull request overview

This PR optimizes VLM (Vision-Language Model) performance on NPU by switching embedding models from CPU to AUTO plugin (GPU/CPU fallback), reducing TTFT by 60-80%. It also addresses NPU's lack of RemoteTensor support and makes AUTO device properties user-configurable.

Changes:

• Changed embedder device from "CPU" to "AUTO" for NPU execution with GPU/CPU fallback priorities
• Added use_intermediate_remote_tensor parameter to disable RemoteTensor usage for NPU
• Refactored device properties handling to use user-configurable AUTO plugin settings instead of hardcoded values

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
tests/python_tests/test_vlm_pipeline.py	Added NPU AUTO config test and formatting cleanup
src/cpp/src/visual_language/pipeline.cpp	Switched embedder device to AUTO for NPU, added AUTO properties configuration, and propagated use_intermediate_remote_tensor flag
src/cpp/src/lm_encoding.hpp	Added use_intermediate_remote_tensor parameter to function signature
src/cpp/src/lm_encoding.cpp	Replaced hardcoded return_remote_tensor with configurable use_intermediate_remote_tensor parameter

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Corrected spelling of 'propeties' to 'properties'.

Suggested change

	auto auto_propeties = utils::pop_or_default<ov::AnyMap>(device_properties, "AUTO", {});
	auto_propeties.insert(ov::device::priorities("GPU", "CPU"));
	auto_propeties.insert(ov::intel_auto::enable_startup_fallback(false));
	device_properties["AUTO"] = auto_propeties;
	auto auto_properties = utils::pop_or_default<ov::AnyMap>(device_properties, "AUTO", {});
	auto_properties.insert(ov::device::priorities("GPU", "CPU"));
	auto_properties.insert(ov::intel_auto::enable_startup_fallback(false));
	device_properties["AUTO"] = auto_properties;

[Copilot]

Corrected spelling of 'auto_propeties' to 'auto_properties'.

Suggested change

	auto auto_propeties = utils::pop_or_default<ov::AnyMap>(device_properties, "AUTO", {});
	auto_propeties.insert(ov::device::priorities("GPU", "CPU"));
	auto_propeties.insert(ov::intel_auto::enable_startup_fallback(false));
	device_properties["AUTO"] = auto_propeties;
	auto auto_properties = utils::pop_or_default<ov::AnyMap>(device_properties, "AUTO", {});
	auto_properties.insert(ov::device::priorities("GPU", "CPU"));
	auto_properties.insert(ov::intel_auto::enable_startup_fallback(false));
	device_properties["AUTO"] = auto_properties;

[Copilot]

The device priorities are hardcoded as 'GPU', 'CPU'. Consider making these configurable through parameters or allowing users to override them via device_properties to maintain consistency with the PR's goal of making properties user-configurable.

Suggested change

	auto_propeties.insert(ov::device::priorities("GPU", "CPU"));
	auto_propeties.insert(ov::intel_auto::enable_startup_fallback(false));
	// Apply default device priorities only if the user did not specify them.
	const std::string priorities_name = ov::device::priorities.name();
	if (auto_propeties.find(priorities_name) == auto_propeties.end() &&
	auto_propeties.find("DEVICE_PRIORITIES") == auto_propeties.end()) {
	auto_propeties.insert(ov::device::priorities("GPU", "CPU"));
	}
	// Apply default startup fallback only if the user did not specify it.
	const std::string startup_fallback_name = ov::intel_auto::enable_startup_fallback.name();
	if (auto_propeties.find(startup_fallback_name) == auto_propeties.end()) {
	auto_propeties.insert(ov::intel_auto::enable_startup_fallback(false));
	}

[AlexanderKalistratov]

insert doesn't modify if key already present

[AlexanderKalistratov]

Please do not merge this PR yet as we need to asses how it is affect energy consumption

[dmatveev]

Lets keep CPU by default but give a way (and document it) on how to put GPU in the config.

[Wovchena]

The config isn't there yet. OVMS are working on fully exposing vision models

[dmatveev]

idk lgtm imo thx!