Skip to content

Commit 1e50354

Browse files
committed
Cut the encoder notes down to what a reader acts on
The raw output note says the bytes differ, who that matters to, that three characters stay escaped so a streamed reply cannot be split, and that switching in either direction needs no migration. The coverage and opt-in bullets on the performance page lose the restatements of the same fallback behaviour described elsewhere on that page and in the reference entry.
1 parent 424cdc5 commit 1e50354

2 files changed

Lines changed: 4 additions & 4 deletions

File tree

docs/troubleshooting/multi-replica.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -335,7 +335,7 @@ Every Socket.IO event in a multi-replica deployment is published through Redis,
335335
Two differences do survive the fallback, neither of which affects a normal deployment:
336336

337337
- **`NaN` and `Infinity` floats in a response body serialize as `null`** instead of raising. Starlette's own encoder is configured with `allow_nan=False`, so a response carrying those values previously failed outright and now goes out with nulls. If you have a custom tool or pipe that can emit them, this is a behaviour change to be aware of rather than a regression.
338-
- **Encoded output carries raw UTF-8 rather than `\uXXXX` escapes.** Both are valid JSON and every client parses them identically, but the exact bytes differ, which matters only if something downstream checksums or string-compares encoded JSON. Three characters are the exception and stay escaped, as the standard library escapes them: line separator (`U+2028`), paragraph separator (`U+2029`) and next line (`U+0085`). Everything that reads a streamed response line by line (Open WebUI itself and any client consuming the OpenAI-compatible stream) counts all three as a line break, so leaving them raw would split a reply mid-stream and silently drop the piece it landed in. They are common in text lifted from PDFs and word processor documents, so the realistic case is a model quoting an uploaded file back. On v0.11.0 the three still went out raw, so if that is the release you are running with this setting on, see [Connection Errors → Missing Text with the Faster JSON Encoder](/troubleshooting/connection-error#missing-text-with-the-faster-json-encoder). This applies to what is stored as well as what is sent: non-ASCII text saved in the database, and in Valkey chunk details, is held raw and takes a little less space. Anything written before you turned the setting on stays readable, turning it back off is equally safe and neither direction needs a migration.
338+
- **Encoded output carries raw UTF-8 rather than `\uXXXX` escapes.** Both parse identically, so this matters only to something downstream that checksums encoded JSON. Three line separator characters stay escaped, since leaving them raw would split a streamed reply. Stored text is held raw too, and neither turning the setting on nor off needs a migration.
339339

340340
**When not to bother.** On a single-worker, single-user instance the saving is real but invisible next to model latency. The setting earns its keep once Socket.IO traffic is crossing Redis between workers or replicas. The one thing that can be felt on a small instance is very long chats, which are encoded in full every time they are saved and decoded again every time they are opened.
341341

docs/troubleshooting/performance.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -224,9 +224,9 @@ See [`ENABLE_COMPRESSION_MIDDLEWARE`](/reference/env-configuration#enable_compre
224224
Open WebUI encodes and decodes JSON constantly: every request body, every API response, every chat saved and opened again, every request sent on to a provider, every chunk of a streamed completion arriving back and every Socket.IO event, including the ones published over Redis when you run multiple workers or replicas. By default all of that goes through Python's standard-library `json` module. Setting `ENABLE_ORJSON=True` switches the whole application to [orjson](https://pypi.org/project/orjson/), a Rust implementation that is several times faster. It is already installed as a dependency, so this is a one-line change.
225225

226226
* **Where the win is**: the Socket.IO encoding path. In clustered deployments, encoding live updates was the single largest cost measured on the workers handling them. Streaming responses benefit too, since every arriving chunk is parsed individually.
227-
* **Where else it shows up**: saving and opening chats. A whole conversation is encoded every time it is saved and decoded again every time it is opened, so the cost follows the length of the chat. The request sent on to the provider carries the same conversation and is encoded on the same path. If code execution or the code interpreter runs on the Jupyter engine, the output of a run comes back on this path as well, so runs that print a lot or generate images land in the chat noticeably sooner.
228-
* **Where it is not**: a single-user instance with ordinary-sized chats. The saving is real but too small to notice against model latency.
229-
* **Why it is opt-in**: orjson is stricter than the standard library. Payloads it rejects (non-string dictionary keys, integers beyond 64 bits, `NaN`/`Infinity` literals) fall back to the standard-library path automatically, so nothing breaks, but the default stays on the standard library to keep behaviour byte-for-byte identical to earlier releases. Encoding that asks for something orjson has no equivalent for, such as indented output or a custom rule for converting values before they are written, falls back the same way, so the request is honoured rather than dropped. The one behaviour change to be aware of: `NaN` and `Infinity` floats in a JSON response now serialize as `null` instead of raising.
227+
* **Where else it shows up**: saving and opening chats, so the cost follows the length of the chat.
228+
* **Where it is not**: a single-user instance with ordinary-sized chats.
229+
* **Why it is opt-in**: anything orjson cannot encode falls back to the standard library automatically, so nothing breaks, but the default keeps behaviour identical to earlier releases. One change to know about: `NaN` and `Infinity` floats serialize as `null` instead of raising.
230230

231231
- **Env Var**: `ENABLE_ORJSON=True`
232232
* *Recommendation*: enable on any Redis-backed multi-worker or multi-replica deployment. Requires a restart. Available from v0.11.0.

0 commit comments

Comments
 (0)