Post-transcription enhancement via cloud LLMs (OpenRouter support, custom dictionary, user-defined prompts) #408

Feature Request

Problem Statement

Vocalinux currently provides excellent offline transcription via Whisper/VOSK, but raw speech-to-text output often contains:

• Domain-specific jargon, brand names, or technical terms that local models misspell
• Grammatical artifacts from spoken language (false starts, filler words, run-on sentences)
• Lack of formatting (no punctuation standardization, capitalization fixes, or structural cleanup)

Users who need polished, publication-ready text currently have to manually edit transcriptions or pipe output through external tools, breaking the seamless "speak and type anywhere" workflow.

Proposed Solution

Introduce an optional post-transcription enhancement layer powered by cloud LLMs, with the following capabilities:

1. Cloud LLM Integration

   • Send raw transcription text to a configurable cloud LLM API for refinement.
   • Support generic OpenAI-compatible endpoints so users can bring their own provider.

2. First-Class OpenRouter Support

   • Native support for OpenRouter as a provider, allowing users to access hundreds of models (Claude, GPT, Llama, Mistral, etc.) with a single API key.
   • Expose model selection (e.g., anthropic/claude-sonnet-4-20250514, openai/gpt-4.1-mini) in settings.

3. Custom User Prompts

   • Allow users to define a system and/or user prompt template that receives the raw transcription.
   • Include a placeholder (e.g., {{transcription}}) that gets substituted at runtime.
   • Provide sensible defaults (e.g., "Fix grammar and punctuation, preserve meaning") while allowing full customization.

4. Dictionary / Vocabulary Support

   • Maintain a per-user custom dictionary (word/phrase → preferred spelling or expansion).
   • The dictionary should be applied before or alongside the LLM pass so that:
     • Proper nouns, technical terms, and acronyms the user cares about are preserved correctly.
     • The LLM prompt can optionally reference the dictionary to enforce consistency.
   • Format: simple JSON or TOML file, editable via UI or config file.

5. Workflow Integration

   • Make the enhancement step opt-in and async so it doesn't block real-time typing.
   • Indicator in the UI when enhancement is pending or complete.
   • Respect user privacy: clear warnings that cloud LLMs send data off-device; default to offline-only unless explicitly enabled.

Use Case

1. User dictates a technical email mentioning "Kubernetes," "Anthropic," and "CUDA."
2. Raw Whisper output: "kubernetes anthropic cud uh see you dee a"
3. Dictionary ensures "Kubernetes," "Anthropic," and "CUDA" are preserved.
4. LLM prompt: "Correct grammar, fix capitalization, and format as a professional email. Transcription: {{transcription}}"
5. Final output inserted at cursor: a clean, professionally formatted email.

Related Issues

• [FEATURE] Expose whisper.cpp initial_prompt in config and settings UI` #393 — [FEATURE] Expose whisper.cpp initial_prompt in config and settings UI (complementary; that feature affects pre-transcription prompting, this request focuses on post-transcription enhancement)

Additional Context

OpenRouter's unified API (https://openrouter.ai/api/v1/chat/completions) is OpenAI-compatible, so supporting generic OpenAI-compatible endpoints would cover OpenRouter automatically. However, explicitly listing OpenRouter in the UI and documentation lowers the barrier for users who want model choice without managing multiple API keys.

Would you be willing to implement this?

• Yes, I would implement this
• Yes, I could help implement this
• No, I just want this feature