whisper.cpp server and ggml-large-v3-turbo-q8_0.bin model bundled together inside a Docker image. Compiled with Vulkan support and without AVX requirement to run on old hardware. Tested using i3-3220 CPU with RX 470 GPU running Fedora Linux.
docker container run \
--detach \
--init \
--device /dev/kfd \
--device /dev/dri \
--read-only \
--tmpfs /root/.cache/mesa_shader_cache \
--restart always \
--name whisper \
--label io.containers.autoupdate=registry \
--publish 8080:8080 \
ghcr.io/kth8/whisper-server-vulkan:latest
Verify if the server is running by going to http://127.0.0.1:8080 in your web browser or using the CLI
curl -s "https://cdn.openai.com/whisper/draft-20220913a/micro-machines.wav" | curl -s -X POST -F "file=@-;filename=audio.wav" http://127.0.0.1:8080/v1/audio/transcriptions