Skip to content

Commit a5a3521

Browse files
committed
Document turning off websocket frame compression
Chat responses stream as one very small frame per token, and the websocket server compresses each of them separately for every subscriber, which costs processor time while saving almost nothing at that size. UVICORN_WS_PER_MESSAGE_DEFLATE=false declines the extension, and the default keeps the previous behaviour so nothing changes until an operator turns it off. The reference entry describes the trade, namely that the frames which did compress well are the rare large ones and even a very long reply is only a few hundred kilobytes uncompressed. The scaling guide, the performance troubleshooting page and the multi-replica guide each carry it beside their existing advice on HTTP response compression, which all three already noted does not apply to websocket traffic, and the production checklist lists it alongside the other per-worker savings.
1 parent dfd7b0f commit a5a3521

4 files changed

Lines changed: 29 additions & 1 deletion

File tree

docs/getting-started/advanced-topics/scaling.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,14 @@ By default every Open WebUI worker compresses its own HTTP responses (JSON API r
163163

164164
WebSocket traffic and streaming chat responses (SSE) are never compressed by this middleware anyway, so disabling it has no effect on the chat streaming path. If nothing in front of Open WebUI compresses responses, the main cost of disabling is a larger first (uncached) page load, several megabytes of JavaScript/CSS, and larger big-JSON payloads (long chat histories, large model lists), which matters mostly on slow or mobile links. See [`ENABLE_COMPRESSION_MIDDLEWARE`](/reference/env-configuration#enable_compression_middleware) for the full trade-off discussion.
165165

166+
WebSocket frames are compressed separately, by the websocket server itself, and that is worth switching off under heavy streaming:
167+
168+
```
169+
UVICORN_WS_PER_MESSAGE_DEFLATE=false
170+
```
171+
172+
Chat responses stream as a very small frame per token, so compressing each one costs processor time for every subscriber and saves almost nothing at that size. The frames that did benefit, a finished message or a set of sources, are a few hundred kilobytes at most even for a very long reply. See [`UVICORN_WS_PER_MESSAGE_DEFLATE`](/reference/env-configuration#uvicorn_ws_per_message_deflate).
173+
166174
#### Pair It with Static Asset Caching at the Proxy
167175

168176
Disabling app-side compression works best when the proxy also **caches the static assets aggressively**, so the "larger first page load" downside effectively disappears: each browser downloads the (proxy-compressed) bundles once and then never asks for them again.

docs/reference/env-configuration.mdx

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8283,6 +8283,13 @@ It is recommended to use the `ENABLE_DB_MIGRATIONS` variable to designate a spec
82838283

82848284
### Cache Settings
82858285

8286+
#### `UVICORN_WS_PER_MESSAGE_DEFLATE`
8287+
8288+
- Type: `bool`
8289+
- Default: `True`
8290+
- Description: Whether the websocket server negotiates per-message compression with the browser. Chat responses arrive as a stream of very small frames, one per token, and compressing each of them costs processor time on the server for every subscriber while saving almost nothing, since there is little to compress in a fragment that size. Setting this to `false` declines the compression extension, so frames go out uncompressed. The large frames that did benefit, a finished message or a set of sources, are only a few hundred kilobytes even for a very long reply, which any network carries without a noticeable delay. Worth turning off on instances under heavy streaming load, where it shows up as measurable processor use. The default preserves the previous behaviour.
8291+
- Persistence: No
8292+
82868293
#### `CACHE_CONTROL`
82878294

82888295
- Type: `str`

docs/troubleshooting/multi-replica.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -309,6 +309,8 @@ While Open WebUI is designed to be stateless with proper Redis configuration, en
309309

310310
By default each Open WebUI worker compresses its own HTTP responses, which profiling shows costs roughly 3–4% CPU per worker, multiplied across all replicas. In multi-replica deployments there is always a load balancer or ingress in front, so enable compression there and disable it in the app with `ENABLE_COMPRESSION_MIDDLEWARE=false`. WebSocket and SSE streaming traffic is never compressed by this middleware, so chat streaming is unaffected. Pair this with proxy-side caching of the content-hashed static bundles under `/_app/immutable/` so they are served from the proxy cache instead of the workers. See [`ENABLE_COMPRESSION_MIDDLEWARE`](/reference/env-configuration#enable_compression_middleware) and [Scaling → Offload HTTP Compression](/getting-started/advanced-topics/scaling#offload-http-compression-to-the-load-balancer).
311311

312+
WebSocket frames are compressed by the WebSocket server rather than that middleware, and under streaming load that is worth switching off with `UVICORN_WS_PER_MESSAGE_DEFLATE=false`. Every replica pays it, since chat responses arrive as one small frame per token and each is compressed separately for every subscriber. See [`UVICORN_WS_PER_MESSAGE_DEFLATE`](/reference/env-configuration#uvicorn_ws_per_message_deflate).
313+
312314
### Use the Faster JSON Encoder
313315

314316
```bash

docs/troubleshooting/performance.md

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,16 @@ By default, Open WebUI compresses HTTP responses (JSON API responses and the sta
220220

221221
See [`ENABLE_COMPRESSION_MIDDLEWARE`](/reference/env-configuration#enable_compression_middleware) for the full trade-off discussion.
222222

223+
#### WebSocket Frame Compression
224+
The HTTP middleware above never touches WebSocket traffic, but the WebSocket server compresses frames on its own, and that is the one worth disabling under streaming load. Chat responses arrive as a very small frame per token, so each is compressed separately, for every subscriber, with almost nothing to gain at that size. Under heavy streaming this shows up as measurable worker CPU.
225+
226+
- **Env Var**: `UVICORN_WS_PER_MESSAGE_DEFLATE=false`
227+
228+
* **What you give up**: the frames that did compress well are the rare large ones, a finished message or a set of sources, and even a very long reply is only a few hundred kilobytes uncompressed, which any network carries without a noticeable delay.
229+
* **Default**: enabled, matching the behaviour before the setting existed, so nothing changes until you turn it off.
230+
231+
See [`UVICORN_WS_PER_MESSAGE_DEFLATE`](/reference/env-configuration#uvicorn_ws_per_message_deflate) for the full description.
232+
223233
#### JSON Encoder
224234

225235
Open WebUI encodes and decodes JSON constantly: every request body, every API response, every chat saved and opened again, every request sent on to a provider, every chunk of a streamed completion arriving back and every Socket.IO event, including the ones published over Redis when you run multiple workers or replicas. By default all of that goes through Python's standard-library `json` module. Setting `ENABLE_ORJSON=True` switches the whole application to [orjson](https://pypi.org/project/orjson/), a Rust implementation that is several times faster. It is already installed as a dependency, so this is a one-line change.
@@ -563,7 +573,8 @@ For multi-user or growing deployments the durable fix is **PostgreSQL**, not SQL
563573
10. **Caching**: `ENABLE_BASE_MODELS_CACHE=True`, `MODELS_CACHE_TTL=300`, `ENABLE_QUERIES_CACHE=True`.
564574
11. **Redis**: Single instance with `timeout 1800` and high `maxclients` (10000+). See [Redis Tuning](#redis-tuning) below.
565575
12. **Compression**: `ENABLE_COMPRESSION_MIDDLEWARE=False` **if** your load balancer / ingress / CDN compresses responses (enable it there instead). Saves ~3–4% CPU on every worker. See [HTTP Response Compression](#http-response-compression).
566-
13. **JSON Encoder**: `ENABLE_ORJSON=True` (v0.11.0+). Cuts the cost of the heaviest JSON work in a clustered deployment: encoding Socket.IO events, parsing streamed provider chunks and saving and opening whole chats. See [JSON Encoder](#json-encoder).
576+
13. **WebSocket Compression**: `UVICORN_WS_PER_MESSAGE_DEFLATE=false`. Streaming sends one tiny frame per token, and compressing each of them costs CPU per subscriber for almost no saving. See [WebSocket Frame Compression](#websocket-frame-compression).
577+
14. **JSON Encoder**: `ENABLE_ORJSON=True` (v0.11.0+). Cuts the cost of the heaviest JSON work in a clustered deployment: encoding Socket.IO events, parsing streamed provider chunks and saving and opening whole chats. See [JSON Encoder](#json-encoder).
567578

568579
#### Redis Tuning
569580

0 commit comments

Comments
 (0)