Inference speed/requirements?

[DavidNTompkins]

Amazing project! I'm starting a training run now and was wondering if there were more details on inference speed on CPU / minimum requirements to run inference? I saw that it's lightning fast on a v100 in the paper - any ideas how it'll perform without GPU?

Ideally, I'm trying to compare to MB-iSTFT-VITS, which claims a ~15x realtime on an i7 CPU. (https://github.com/MasayaKawamura/MB-iSTFT-VITS).

Thanks either way!

[p0p4k]

There is another repo by Fenrir with the istft mb stuff. I wanna add it to this repo or make another extension if it is good enough / worth the efforts.

[DavidNTompkins]

Thanks! I might give that a try - best of both worlds

[bzp83]

if it helps, I trained 2 models, one with this repo and the other with fenrir's for a few epochs (enough to output something) to test, the model from this repo has a rtf of ~0.17x and fenrir's with ms_istft is ~0.04x. I exported both to onnx and ran on cpu, my cpu is an i9-12950HX.

Both outputs were very close checking by ear (still low quality due to low epochs) but it was impossible to tell the difference.

Now I need to do a proper training to check quality, but I wanted to train with 44100hz and I'm struggling to find the proper configs.

[w11wo]

My colleague @DavidSamuell has conducted inference speed tests on CPU, GPU, ONNX CPU, and ONNX GPU. You can find the full results here.

Model: Pytorch CPU: 2.3481 seconds
Model: Pytorch GPU: 0.1600 seconds
Model: ONNX CPU: 2.1909 seconds
Model: ONNX GPU: 0.1367 seconds