Feature request - Support CUDA 12.x with onnxruntime-gpi 1.21.x

[asusdisciple]

🚀 Feature

Support CUDA and onnxruntime to run the silero models on GPU. It works on CPU but fails on GPU currently.

Motivation

The motivation is to support the newest available infrastructure.

Pitch

Provide the onnx model which makes this possible. Unfortunately i can not export it myself alone with just the jit model.

Alternatives

Not really there, since no everyone can change the environment to support this.

Additional context

[mwiktorczyk]

Is there any chance to move onnxruntime as an extra dependency? So something like this:

dependencies = [
"torch>=1.12.0",
"torchaudio>=0.12.0"
]

[project.optional-dependencies]

onnx-cpu = [
"onnxruntime>=1.16.1",
]

onnx-gpu = [
"onnxruntime-gpu>=1.16.1",
]

[snakers4]

Technically, code in the repo supports both PyTorch and ONNX-runtime, so we would need to provide a transparent way to require both of these.

I believe you could have optional dependencies like this:

[project.optional-dependencies]
gui = ["PyQt5"]
cli = [
  "rich",
  "click",
]

and then install like pip install package[gui] or pip install package[click].

But before doing something like this, I would wait for a more broad confirmation that this feature is required.

[mwiktorczyk]

The problem lies with the onnxruntime and onnxruntime-gpu libraries, because from pip’s point of view these two libraries are treated as one and the same package. As a result, it’s impossible to install both at the same time, since one will overwrite the other. Consequently, GPU support cannot be used.

[Purfview]

Why do you need to run it on GPU? Silero ran much slower on GPU [RTX 4070] than on CPU when I tested it.