OpenVINO support

[tazz4843]

Possible replacement for removed OpenCL support? Looks promising

[thewh1teagle]

@tazz4843

Looks interesting. Does it works seamlessly on macOS/Linux/Windows just like with Vulkan?
Do we need to install something special when using it? (not when building)
Also I'm not sure if it works with normal ggml models

I'm pretty sure that it requires installing openvino runtime on the target machine. With vulkan it works without since the GPUs drivers comes with the runtime

[newfla]

@tazz4843 LGTM. Do you think it is possible to release a new version on crates.io with this pr merged?

[tazz4843]

I was going to test this one last time before merging but got stuck into a massive rabbit hole of trying to get OpenVINO on my Framework 16 running Fedora (long story short it turns out OpenVINO is only supported on Fedora 42 which isn't released until April 🥴)

I'll test on my Arch desktop when I have more free time unless one of you two would like to test and see if it still works.

[tazz4843]

Windows error seems to have randomly appeared and without a Windows system to test on I've just removed it from the required matrix.

Nonetheless I still can't get OpenVINO working because it randomly errors out and I have no clue why:

whisper_ctx_init_openvino_encoder_with_state: loading OpenVINO model from '/home/niko/data/whisper.cpp/models/ggml-base-encoder-openvino.xml'
whisper_ctx_init_openvino_encoder_with_state: first run on a device may take a while ...
whisper_openvino_init: path_model = /home/niko/data/whisper.cpp/models/ggml-base-encoder-openvino.xml, device = CPU, cache_dir = /home/niko/data/whisper.cpp/models/ggml-base-encoder-openvino-cache
whisper_ctx_init_openvino_encoder_with_state: OpenVINO model loaded
Progress callback: 0%
in openvino encode inference execution routine: exception: Check 'shape_strides[i] <= m_strides[i]' failed at src/inference/src/dev/make_tensor.cpp:139:
shape stride: 1920000, stride: 960000

This also causes the transcript to return instantly and it's always either just [Music] or whatever the initial prompt was, verbatim.