[FORMATS] Most important export formats?

[snakers4]

As everybody knows, the VAD now exists as an ONNX model (onnx-runtime) and as a jit (PyTorch TorchScript) model.

Can you please shorlist the remaining most important export formats, i.e. TF, OpenVINO, etc?

We need some data / votes to make a shortlist.

To make it simpler - I will just create a few asnwers with options.

Please feel free to add more and upvote the most necessary ones.

[snakers4]

OpenVINO

[snakers4]

TensorFlow (which format exactly? there many varieties)

[snakers4]

#705

[WilliamTambellini]

onnx, and perhaps gguf. Tks

[snakers4]

onnx

we have onnx

and perhaps gguf. Tks

Isn't this format for huge (100B) LLMs?

[WilliamTambellini]

Isn't this format for huge (100B) LLMs?

oh no, it s now for any/most NN: silero-vad v5 has already been added a long time ago:
https://github.com/ggml-org/whisper.cpp/blob/master/models/for-tests-silero-v5.1.2-ggml.bin
FYI:
ggml-org/whisper.cpp#3450

[AlexErrant]

Some notes from the person (danbev) who converted Silero v5 to ggml: https://github.com/danbev/learning-ai/blob/main/notes/vad.md

[jwijffels]

+1 gguf

[WilliamTambellini]

done:
ggml-org/whisper.cpp#3524
@snakers4

[snakers4]

@WilliamTambellini

done: ggml-org/whisper.cpp#3524 @snakers4

Not sure yet what to do with this, though. We could probably use the converted model somehow and / or add it to our basic release schedule.

Optionally removing PyTorch and / or ONNX runtime dependency is enticing (probably it wiil make the VAD even faster / consume even less resources?), but whisper.cpp does not have python bindinds as far as I see. So probably we could extract the VAD code from the repo, minimize it, compile it with minimal examples for major platforms.

Not sure we could do this, since we do no use C++ daily. Also doing this would imply setting up the CI properly, so that the builds are auto-generated for all major platforms (Linux, Windows, ARM, etc) on new VAD releases.

Seems like a lot of work (for us).

https://github.com/ggml-org/ggml/tree/master/examples/python - looks dated