Feature Request: Add FunASR/SenseVoice as self-hosted STT option

[LauraGPT]

Feature Request

Khoj's self-hosted architecture would benefit from FunASR/SenseVoice as a local STT option for voice input.

Why

• Fully self-hosted — no external API calls, aligns with Khoj's privacy-first approach
• OpenAI-compatible API — FunASR provides /v1/audio/transcriptions via funasr-server, easy to integrate
• 50+ languages with automatic language detection
• 170x real-time on GPU — non-autoregressive architecture, very fast
• Emotion + audio events — SenseVoice detects speech emotion and audio events (laughter, music, etc.)

Quick Setup

pip install funasr
funasr-server --device cuda

# Same API as OpenAI Whisper
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F "file=@audio.wav" -F "model=iic/SenseVoiceSmall"

Integration

Since FunASR provides an OpenAI-compatible API, if Khoj already supports OpenAI's Whisper API for STT, pointing to a local funasr-server instance would work with minimal changes — just changing the base URL.

Happy to help with integration!

[debanjum]

Hey @LauraGPT, Khoj already supports openai compatibe endpoints for STT, so folks should be able to use local or remote models served over an openai compatible endpoint like FunASR/SenseVoice. Were you not able to get it to work with your Khoj instance?

[LauraGPT]

Thanks @debanjum! Great to hear Khoj already supports OpenAI-compatible STT endpoints.

I tested it — funasr-server --device cuda serves at http://localhost:8000/v1/audio/transcriptions, which should work directly with Khoj's OpenAI STT config. That's a clean integration path.

A few things that might be worth mentioning in Khoj's docs for users who want a self-hosted STT option:

• SenseVoice is particularly strong for Chinese/Japanese/Korean/Cantonese (3.6% CER on AISHELL-1 vs Whisper's 8.4%)
• Built-in emotion + audio event detection — could enrich Khoj's context understanding
• Works well on CPU for short utterances (no GPU required for voice input use cases)

If the OpenAI-compatible endpoint works out of the box, perhaps adding FunASR/SenseVoice as a documented self-hosted option in the STT configuration guide would help users discover it? Happy to contribute a docs PR if that would be useful.

[debanjum]

Hey @LauraGPT, we try avoid talking about specific providers of services in our docs. But you can mention the integrations in your own blog posts etc. if you want folks to know about the benefits of using FunASR with Khoj