feat: add optional local transcript cleanup after speech recognition

Adds an opt-in, self-hosted text model that corrects grammar, punctuation,
capitalization, and paragraph breaks on a final transcript. It runs on the
gateway machine after recognition, sees only recognised text (never audio),
and needs no Voca account and no internet access once its model is installed.

Off by default, and off until an operator installs a model and enables it.

The governing rule is that cleanup can improve a transcript and never lose
one. The deterministic writing-style result is computed first and
independently; every way cleanup can fail — no model, unsupported language,
input over the ceiling, runtime busy, timed out, malformed answer, or an edit
the checks refuse — returns that exact string, with a bounded reason. A
recognition that succeeded stays a success: cleanup failure is metadata on a
200, never an error a client has to handle. Cancellation propagates rather
than being swallowed as a tidy fallback.

app/cleanup/ holds the whole feature:

- service.py is the single finalization every entry point shares, so sessions,
  one-shot requests, and streaming cannot drift into different decisions.
- validation.py refuses a candidate that changes a protected span, a digit run,
  a negation, a weekday or month, the writing system, or more of the text than
  an edit budget allows, and rejects wrappers, refusals, leaked reasoning, and
  truncated generations.
- prompts.py serialises the transcript as JSON data inside a user message, so a
  dictated "ignore the previous instructions" is content to preserve.
- transport.py owns a raw loopback socket: no URL to redirect, no proxy
  inherited, a bounded body, and a close that actually stops the backend.
- worker.py/host.py launch and own a llama-server via argv on an unpublished
  ephemeral port with a credential of its own; manager.py resolves environment
  overrides against saved UI choices, admits one inference at a time, and
  unloads an idle worker without ever touching the speech engine.
- catalog.py is a separate namespace with its own pin file. A cleanup artifact
  with no pinned revision and SHA-256 cannot be installed at all, and never
  appears as a selectable speech engine.

Wiring:

- Sessions take cleanup: off|conservative|inherit, snapshotted at creation, and
  return original_transcript plus a redacted cleanup block. finish/retry answer
  from storage without a second model call.
- /v1/audio/transcriptions takes a multipart cleanup field defaulting to off,
  keeps the {"text": ...} body unchanged, and reports status in optional
  X-Voca-Cleanup-* headers.
- /v1/stream runs exactly one cleanup pass, after the streaming lock and the
  engine lease are released. Partials are never rewritten.
- New GET /v1/capabilities reports supported modes and languages, keeping
  offered languages separate from evaluated ones.
- Raw style is never corrected, whatever a request asks for.

Also closes the read-then-update race around concurrent finish: the state test
now lives inside the statement, so exactly one caller starts a transcription
and a session deleted or re-uploaded mid-flight is neither resurrected nor
overwritten by the stale job.

Storage migrates additively and idempotently; existing rows and transcripts are
untouched and read back as cleanup off with no manufactured original.

The shipped candidates are the upstream-published Qwen3 0.6B and 1.7B Q8_0
GGUFs. Qwen publishes no smaller quantization in its own repositories, and a
community Q4_K_M is a separate artifact whose bytes the upstream card does not
vouch for, so first-party provenance won over file size. evaluated_languages is
empty for both: no benchmark has been run, so nothing claims a tested language.

Author: Mr-Sunglasses

.env.example

@@ -198,6 +198,47 @@ VOCAGATEWAY_PUBLISH_PORT=8765
 # local development.
 #VOCAGATEWAY_DEBUG=true
 
+# ----------------------------------------------------------------------------
+# Transcript cleanup (optional, off by default)
+# ----------------------------------------------------------------------------
+# A small text model that fixes grammar, punctuation, and casing *after*
+# recognition, on the recognised text only. Audio never reaches it. Whenever it
+# cannot finish safely you get exactly the transcript you would have got with
+# the feature off — it can improve a result, never lose one.
+#
+# In Compose it needs the opt-in `cleanup` sidecar, because the gateway image
+# ships no text model and no llama.cpp runtime:
+#
+#   VOCAGATEWAY_CLEANUP_MODEL_DIR=$HOME/.local/share/vocagateway/models/cleanup \
+#   VOCAGATEWAY_CLEANUP_MODEL_FILE=/models/llama.cpp/Qwen3-0.6B-Q8_0.gguf \
+#   VOCAGATEWAY_CLEANUP_ENDPOINT=cleanup:8080 \
+#   VOCAGATEWAY_CLEANUP_API_KEY=$(openssl rand -hex 24) \
+#   docker compose --profile cleanup up -d
+#
+# Leaving these unset is not the same as setting them to off: unset means "the
+# WebUI's saved choice decides", while a set value locks the toggle in the UI.
+#VOCAGATEWAY_CLEANUP_ENABLED=true
+#VOCAGATEWAY_CLEANUP_MODE=conservative
+#VOCAGATEWAY_CLEANUP_MODEL=cleanup:qwen3-0.6b
+#VOCAGATEWAY_CLEANUP_TIMEOUT_SECONDS=5
+
+# Languages cleanup is allowed to run for. Defaults to the selected model's own
+# list. These are languages the feature is *offered* for, not ones an
+# evaluation has signed off — the WebUI reports the two separately.
+#VOCAGATEWAY_CLEANUP_LANGUAGES=en,hi,hinglish_roman
+
+# Address of the sidecar, and the credential the gateway presents to it. Only
+# loopback, private ranges, and bare Compose service names are accepted; a
+# routable address is refused at startup, because "runs on your gateway" has to
+# stay true. A client's bearer token is never forwarded here.
+#VOCAGATEWAY_CLEANUP_ENDPOINT=cleanup:8080
+#VOCAGATEWAY_CLEANUP_API_KEY=
+
+# Compose-only inputs for the sidecar itself.
+#VOCAGATEWAY_CLEANUP_IMAGE=ghcr.io/ggml-org/llama.cpp:server
+#VOCAGATEWAY_CLEANUP_MODEL_DIR=./models
+#VOCAGATEWAY_CLEANUP_MODEL_FILE=/models/model.gguf
+
 
 # ============================================================================
 # 6. Listener inside the container

README.md

@@ -76,6 +76,7 @@ contract is in [configuration.md](docs/configuration.md).
 - [WebUI](#webui)
   - [Fast model guide](#fast-model-guide)
 - [Model download integrity](#model-download-integrity)
+- [Transcript cleanup](#transcript-cleanup) — optional local grammar and punctuation repair
 - [Engine selection](#engine-selection)
 - [Configuration](#configuration) — every `VOCAGATEWAY_*` variable and its default
 - [Listener and network access](#listener-and-network-access)
@@ -490,14 +491,71 @@ change; `--only` explicitly refreshes the matching model or family.
 uv run scripts/harvest-model-pins.py                     # newly added models
 uv run scripts/harvest-model-pins.py --only whisperkit:  # refresh one family
 uv run scripts/harvest-model-pins.py --refresh           # refresh everything
+uv run scripts/harvest-model-pins.py --cleanup           # transcript-cleanup models
 ```
 
+Transcript-cleanup artifacts have their own catalog and their own pin file,
+[`app/cleanup_model_pins.json`](app/cleanup_model_pins.json). They are held to a
+stricter rule than speech models: an entry with no pinned revision and digest
+cannot be installed **at all**, because unverifiable weights must never reach a
+runtime launch.
+
 Each revision and its digests are written as one snapshot. If the complete
 snapshot cannot be collected, the command fails and preserves the previous
 record rather than combining a new revision with stale hashes. Review the
 resulting diff as carefully as code. A changed digest means the upstream bytes
 changed, and the commit message should say why.
 
+## Transcript cleanup
+
+Optional. Off by default, and off until you install a model and turn it on.
+
+A small text model runs **after** speech recognition and fixes grammar,
+punctuation, capitalization, and paragraph breaks while keeping what you said.
+It runs on your gateway, needs no Voca account, and needs no internet access
+once the model is installed. Audio never reaches it — only the recognised text
+does.
+
+The design rule is that it can improve a transcript and never lose one. Every
+way it can fail — no model, wrong language, text too long, busy, timed out, or
+an edit the safety checks refuse — returns exactly the transcript you would have
+got with the feature switched off, plus a bounded reason saying why.
+
+**Setup.** Settings → Transcript cleanup → install a model → **Load model now**
+→ tick *Correct transcripts by default*. Natively the gateway launches and owns
+a `llama-server` on loopback (install llama.cpp, or set
+`VOCAGATEWAY_CLEANUP_BINARY`). Under Compose it is an opt-in sidecar that
+publishes no port:
+
+```sh
+docker compose --profile cleanup up -d
+```
+
+**What it will not do.** It does not translate, summarise, answer questions,
+add content, or invent facts. **Raw is never corrected**, whatever a request
+asks for. And it cannot fix a word the speech model misheard: it only sees
+text, so a wrong word that reads as a sentence stays. Filler and stutter removal
+are deliberately out — repetition often carries meaning.
+
+**Languages.** A transcript left on `auto` is only corrected when its writing
+system names one supported language on its own. Latin script does not, so ask
+for `en` explicitly rather than relying on detection. The WebUI reports *offered*
+and *tested* languages separately: a language is offered because a model claims
+it, and tested only once an evaluation has signed it off.
+
+**Clients.** Sessions and `/v1/stream` take
+`cleanup: "off" | "conservative" | "inherit"`;
+`POST /v1/audio/transcriptions` takes a multipart `cleanup=off|conservative`
+that defaults to `off` and answers in the unchanged `{"text": ...}` shape, with
+status in optional `X-Voca-Cleanup-*` headers. `GET /v1/capabilities` says what
+this gateway supports; a gateway that predates the feature answers `404`, and a
+client must then omit the new fields. Session responses carry
+`original_transcript` beside `transcript` whenever cleanup was asked for, under
+the same retention rules, so the recognised text is always recoverable.
+
+See [configuration.md](docs/configuration.md#transcript-cleanup) for the
+`VOCAGATEWAY_CLEANUP_*` variables and both deployment shapes.
+
 ## Engine selection
 
 The `auto` engine preference uses the first runnable option in this order:

app/admin_queries.py

@@ -2,11 +2,12 @@
 
 from importlib import util as importlib_util
 from types import MappingProxyType
-from typing import Any
+from typing import Any, cast
 
 from app import schemas
 from app.build_info import current_commit
 from app.catalog import catalog_source_url, language_names, recommended_ids
+from app.cleanup import catalog as cleanup_catalog
 from app.config import Settings
 from app.context import BOOTSTRAP_TOKEN_ID, TOKEN_FILE_HINT, VERSION, GatewayContext
 from app.engine_state import active_model_path, available_engines, engine_id
@@ -15,6 +16,9 @@
 from app.system import SystemInfo, detect_system
 
 PYTHON_PACKAGE_PATH = "Python package"
+INSTALLED_STATE = "installed"
+DOWNLOADING_STATE = "downloading"
+NOT_INSTALLED_STATE = "not_installed"
 PYTHON_ENGINE_INSTALL_HINT = "Install vocagateway[engines] or use the Docker image"
 # One engine paired with the single runtime it needs.
 _EngineRuntime = tuple[str, schemas.DependencyStatus]
@@ -318,7 +322,7 @@ def build_custom_entry(self, custom: Any) -> schemas.AdminModelEntry:
             family="Custom Whisper",
             description="User-provided local model.",
             source="Local file",
-            state="installed",
+            state=INSTALLED_STATE,
             active=custom.path == self.active_path,
             offloaded=bool(
                 custom.path == self.active_path
@@ -355,16 +359,16 @@ def filter_by_criteria(
         return matching
 
     def _resolve_state(self, download: Any, inst: Any) -> _ModelState:
-        if download and download.status == "downloading":
+        if download and download.status == DOWNLOADING_STATE:
             progress = None
             if download.total_bytes:
                 progress = round(download.downloaded_bytes / download.total_bytes, 4)
-            return "downloading", progress, None
+            return DOWNLOADING_STATE, progress, None
         if inst:
-            return "installed", None, None
+            return INSTALLED_STATE, None, None
         if download and download.status == "failed":
-            return "not_installed", None, download.error
-        return "not_installed", None, None
+            return NOT_INSTALLED_STATE, None, download.error
+        return NOT_INSTALLED_STATE, None, None
 
 
 async def status_payload(ctx: GatewayContext) -> schemas.AdminStatusResponse:
@@ -422,7 +426,7 @@ def filtered_model_entries(
     helper = _ModelEntryHelper(ctx)
     entries = model_entries(ctx)
     if installed_only:
-        entries = [entry for entry in entries if entry.state == "installed"]
+        entries = [entry for entry in entries if entry.state == INSTALLED_STATE]
     entries = helper.filter_by_criteria(
         entries,
         language=language,
@@ -453,6 +457,96 @@ def token_entries(ctx: GatewayContext) -> list[schemas.DeviceTokenEntry]:
     return entries
 
 
+def cleanup_config(ctx: GatewayContext) -> schemas.CleanupConfigResponse:
+    """The cleanup block, as the settings page and a diagnostics bundle see it.
+
+    Carries no runtime address and no executable path: those are operator-only
+    settings, and a redacted bundle attached to a bug report has no reason to
+    describe the deployment's internal topology.
+    """
+    manager = ctx.cleanup
+    if manager is None:
+        return schemas.CleanupConfigResponse()
+    report = manager.status()
+    return schemas.CleanupConfigResponse(
+        enabled=report.enabled,
+        mode=cast(Any, report.mode),
+        model_id=report.model_id,
+        model_label=report.model_label,
+        model_installed=report.model_installed,
+        runtime_available=report.runtime_available,
+        managed=report.managed,
+        state=cast(Any, report.state),
+        timeout_seconds=report.timeout_seconds,
+        languages=list(report.languages),
+        evaluated_languages=list(report.evaluated_languages),
+        idle_unload_enabled=report.idle_unload_enabled,
+        idle_unload_minutes=report.idle_unload_minutes,
+        locked_settings=list(report.locked_settings),
+        detail=report.detail,
+    )
+
+
+def cleanup_model_entries(ctx: GatewayContext) -> list[schemas.CleanupModelEntry]:
+    """The cleanup catalog with each artifact's install state and provenance.
+
+    Deliberately its own list. A cleanup model must never appear among the
+    speech models: it cannot transcribe anything, and offering it as an engine
+    would be an invitation to select it as one.
+    """
+    manager = ctx.cleanup
+    active = manager.model_id if manager else None
+    return [_cleanup_entry(ctx, model, active) for model in cleanup_catalog.CLEANUP_CATALOG]
+
+
+def _cleanup_entry(
+    ctx: GatewayContext, model: cleanup_catalog.CleanupModel, active: str | None
+) -> schemas.CleanupModelEntry:
+    manager = ctx.cleanup
+    download = manager.models.download_state(model.id) if manager else None
+    installed = manager is not None and manager.models.installed_path(model.id) is not None
+    state, progress, error = _cleanup_state(download, installed=installed)
+    return schemas.CleanupModelEntry(
+        id=model.id,
+        label=model.label,
+        description=model.description,
+        runtime=model.runtime,
+        size_bytes=model.size_bytes,
+        minimum_ram_gb=model.minimum_ram_gb,
+        upstream_model=model.upstream_model,
+        quantization=model.quantization,
+        conversion_source=model.conversion_source,
+        chat_template_source=model.chat_template_source,
+        license_name=model.license_name,
+        license_notice=model.license_notice,
+        source_url=model.source_url,
+        revision=model.revision,
+        sha256=model.sha256,
+        installable=model.installable,
+        languages=list(model.candidate_languages),
+        evaluated_languages=list(model.evaluated_languages),
+        state=state,
+        active=model.id == active,
+        progress=progress,
+        downloaded_bytes=download.downloaded_bytes if download else None,
+        total_bytes=download.total_bytes if download else None,
+        error=error,
+    )
+
+
+def _cleanup_state(download: Any, *, installed: bool) -> _ModelState:
+    if download and download.status == DOWNLOADING_STATE:
+        progress = None
+        if download.total_bytes:
+            progress = round(download.downloaded_bytes / download.total_bytes, 4)
+        return DOWNLOADING_STATE, progress, None
+    if installed:
+        return INSTALLED_STATE, None, None
+    if download and download.status == "failed":
+        return NOT_INSTALLED_STATE, None, download.error
+    return NOT_INSTALLED_STATE, None, None
+
+
 def config_response(ctx: GatewayContext) -> schemas.ConfigResponse:
     rc = ctx.engine_manager.runtime_config if ctx.engine_manager else None
     if rc:
@@ -471,6 +565,7 @@ def config_response(ctx: GatewayContext) -> schemas.ConfigResponse:
             cpu_threads=rc.cpu_threads,
             idle_offload_enabled=rc.idle_offload_enabled,
             idle_offload_minutes=rc.idle_offload_minutes,
+            cleanup=cleanup_config(ctx),
         )
     return schemas.ConfigResponse(
         engine="custom",
@@ -487,4 +582,5 @@ def config_response(ctx: GatewayContext) -> schemas.ConfigResponse:
         cpu_threads=0,
         idle_offload_enabled=False,
         idle_offload_minutes=DEFAULT_IDLE_OFFLOAD_MINUTES,
+        cleanup=cleanup_config(ctx),
     )

app/cleanup/__init__.py

@@ -0,0 +1,11 @@
+"""Optional, local, conservative transcript cleanup.
+
+Off by default, and off unless an operator installs a model and turns it on.
+Audio never reaches this package — only recognised text does — and nothing here
+can turn a successful transcription into a failed one.
+
+Import from the modules directly rather than from this package: `base` for the
+vocabulary, `service` for the single decision every entry point shares,
+`manager` for the runtime that owns the process, `validation` for the checks
+that decide whether a candidate is safe to insert.
+"""