Willow-inference-server (WIS) allows us to use local CUDA cores to translate speech to text and many other services. This project is intended to move the local functionality to Cloudflare Worker AI and its openAI whisper model.
WIS does many different things. The completed first goal of this project was porting over the willow API endpoint as a drop in replacement for the now inaccessible default "test" endpoint of https://infer.tovera.io/api/willow. This allows speech to be converted to text and then be returned to the ecosystem while removing the requirement for a local server install with CUDA cores.
Speech is sent out from the device to a chosen endpoint using the options as defined in the Willow Application Server (WAS). Currently only PCM is supported. The PCM option is found under the advanced settings in WAS.
Cloudflare's Workers provide us with a free (limited calls) hosted alternative environment.
-
Sign up for a Cloudflare account. This is currently free and allows for up to 1000 calls a day
-
Deploy the code to your own worker
- Note: You must Enable AI Binding to run machine learning models
-
In your Willow Application Server, modify the Willow Inference Server Speech Recognition URL to point to your new published endpoint