Ollama openvino integration

[FionaZZ92]

Add new module of Ollama-OV which integrate OpenVINO GenAI as backend engine of Ollama to accelerate LLM inference on Intel platforms (CPU/iGPU/dGPU/NPU).

• current feature: LLM serving with OV genAI 2025RC2 supported model with single request/response token inference.
• will add:

1. streaming token generation - Done.
2. update to 2025RC2. (Current 2025.2.0.0.dev20250320) - Done

[ilya-lavrenov]

current feature: LLM serving with OV genAI 2025RC1 supported model with single request/response token inference

will it be beneficial in future to reuse OpenVINO GenAI Cont Baching pipeline which is faster in case of multiple requests?

[ilya-lavrenov]

could you please also add a single github actions workflow to check that integration is working?

[FionaZZ92]

current feature: LLM serving with OV genAI 2025RC1 supported model with single request/response token inference

will it be beneficial in future to reuse OpenVINO GenAI Cont Baching pipeline which is faster in case of multiple requests?

Our current integration just adopt ov::genai::LLMPipeline basic API integration, and the streaming generate with callback func is WIP. All these two workload are single client single server mode. I do not sure the invoke condition with CB under this mode, and optimization necessary. I think situation should be all aligned with current genAI benchmark.

For multi-requests, yes, that should be have potential benefits. It will require to wrapper interface for ov::genai::ContinuousBatchingPipeline. I think it should be similar way of OVMS. I won't expect this feature in nearly version with 25.1 FRC, unless has customer request directly relevant to it.

[FionaZZ92]

could you please also add a single github actions workflow to check that integration is working?

I think it is not make sense to add .github action in this folder cause github will think it is subrepo require to have hyperlink with the official repo. It means we have to maintain another official repo. If you really want to have an action check, do you think we add this submodule source compiling & test to here is enough? I think no need to download all model to test, that's too heavy.

[zhaohb]

could you please also add a single github actions workflow to check that integration is working?

Hi @ilya-lavrenov We have added an action workflow and verified that the integration is working, please review it.

[FionaZZ92]

Able to merge it now?

[FionaZZ92]

@alvoron Learn that you are the one of owners for openvino_contrib, could you please help to review this PR. And help to approve merge if no more change need.