@@ -134,6 +134,38 @@ export TORCH_CUDA_ARCH_LIST="8.0 9.0 10.0 12.0 12.1a"
134134
135135After applying the fix, restart `wlk`. Incoming streams will now compile kernels targeting `sm_121a` without crashing.
136136
137+ ## CPU throughput
138+
139+ ### Transcription falls further and further behind on CPU
140+
141+ If the transcript lags more the longer a session runs, the machine is spending
142+ more time on ASR than the stream produces audio. Check the compute time shown
143+ in the web UI against elapsed time: once it approaches 100%, the backlog grows
144+ without bound and no amount of waiting clears it.
145+
146+ Each inference pass costs roughly the same regardless of how much *new* audio it
147+ covers, so when chunks are short most of that work re-encodes audio the previous
148+ pass already saw. `--asr-coalesce-min-s` waits for more new audio before running
149+ a pass, which cuts the number of passes at the cost of updating the transcript
150+ less often:
151+
152+ ```bash
153+ wlk --model base --asr-coalesce-min-s 0.75
154+ ```
155+
156+ Off by default. Two caveats worth knowing before you tune it:
157+
158+ - The useful value depends on how large the incoming chunks already are, and the
159+ response is a step rather than a gradual curve: a threshold below the typical
160+ chunk size does almost nothing, and just above it can halve the passes. Start
161+ near your chunk size and measure.
162+ - Words reach the screen in larger, less frequent updates, and the first word of
163+ an utterance arrives later. `--asr-coalesce-max-s` caps how much audio may be
164+ held back, which bounds that delay.
165+
166+ If compute time is comfortably below elapsed time and the transcript still lags,
167+ this is not the right fix.
168+
137169---
138170
139171Need help with another recurring issue? Open a GitHub discussion or PR and reference this document so we can keep it current.
0 commit comments