Skip to content

Commit 7c9b289

Browse files
committed
Lead the essentials context section with Context Compaction
The context management section still said Open WebUI ships no way to handle a full context window and sent everyone straight to writing a filter Function. Context Compaction has covered the common case for a while now, so a reader following that page was being told to build something that already exists. It now leads with compaction: what it does, that an administrator has to turn it on first and where, the threshold to set, and the two limits worth knowing before relying on it, that crossing the threshold does not guarantee it ran and that the token figure is an estimate where the provider reported none. Filter Functions stay as the answer for any other policy. The troubleshooting page also gains the Retained Messages setting, which the admin panel has but its list was missing.
1 parent 2587213 commit 7c9b289

2 files changed

Lines changed: 17 additions & 5 deletions

File tree

docs/getting-started/essentials.mdx

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -131,18 +131,29 @@ After enough back-and-forth you will eventually see:
131131
This error comes from your model provider, not from Open WebUI. Every time you send a message, the *entire* conversation (system prompt, all previous turns, attached files, tool call results, and your new message) is sent as the "prompt." When the sum exceeds the model's context window, the provider rejects the request.
132132
:::
133133

134-
Open WebUI intentionally does not ship a built-in trimmer, because:
134+
### The built-in answer: Context Compaction
135135

136-
- Every model uses a **different tokenizer** (GPT, Claude, Gemini, GLM, Qwen all differ).
137-
- Every model has a **different context window** (8k to 1M+).
138-
- Every deployment wants a **different policy** (trim by tokens, by turns, by message count, drop attachments first, summarize older messages, etc.).
136+
Open WebUI ships **Context Compaction** for exactly this. Once a chat's estimated context passes a token threshold, a model summarizes the older messages, and the conversation continues from that summary plus the most recent messages, so a long chat keeps going instead of hitting the wall. It runs server-side and changes only what is sent to the model on that turn. Your chat is still stored and displayed in full, and nothing is deleted.
139137

140-
There is no single correct answer. The supported approach is to install a **filter Function** that trims the conversation on your terms.
138+
It is **off by default**. An administrator turns it on in **Settings > Admin > Experience > Interface**, where the token threshold (default `80000`), the model that writes the summaries, the share of recent messages kept verbatim and an optional custom summary prompt are all configurable. Set the threshold below your model's real context window.
139+
140+
Two things to know before you lean on it:
141+
142+
- Crossing the threshold does not guarantee it ran. The cut has to land on a user message, so a chat that is one enormous exchange rather than many turns is sent uncompacted. That happens silently and is retried on the next turn, so compaction is a relief valve rather than a hard ceiling.
143+
- The token figure is an estimate wherever your provider did not report real usage, so read it as a gauge rather than an exact count.
144+
145+
With it enabled, `/compact` in the chat input compacts the conversation right away instead of waiting for the threshold.
146+
147+
### When you need a different policy
148+
149+
Compaction is the summarize-and-replace strategy. For a hard cap on chat length, attachments dropped before dialogue, tool results trimmed first or a different window per model, install a **filter Function** instead. Open WebUI ships no blind trimmer, because tokenizers, context windows and the right policy all differ per model and per deployment.
141150

142151
:::tip Quick fix
143152
Community filters for most common policies already exist and can be installed with one click. If none fits, the code is short enough to copy and adapt. See the full guide including a minimal "newest N turns" filter: [Troubleshooting: Context Window / Prompt Too Long](/troubleshooting/context-window).
144153
:::
145154

155+
More detail: [Context Compaction](/troubleshooting/context-window#the-built-in-option-context-compaction) covers the checkpointing, the fallbacks when the summarizer fails, and the `context_usage` API for showing how close a chat is to compacting.
156+
146157
---
147158

148159
## Basic RAG

docs/troubleshooting/context-window.mdx

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,7 @@ It is **disabled by default**. An administrator enables and tunes it in **Settin
5757
- **Context Compaction**: the on/off toggle ([`ENABLE_CONTEXT_COMPACTION`](/reference/env-configuration#enable_context_compaction)).
5858
- **Context Compaction Model**: the model that writes the summaries ([`CONTEXT_COMPACTION_MODEL`](/reference/env-configuration#context_compaction_model)). Leave it on **Current Model** to follow the task model as before.
5959
- **Token Threshold**: the estimated context size, in tokens, above which older messages are compacted (default `80000`, [`CONTEXT_COMPACTION_TOKEN_THRESHOLD`](/reference/env-configuration#context_compaction_token_threshold)). Set it below your model's real window. This is the global default: an individual model can set its own **Context Compaction Threshold** (`compact_token_threshold`) in its advanced parameters, either lower or higher, bounded by the **Token Cap** ([`CONTEXT_COMPACTION_TOKEN_CAP`](/reference/env-configuration#context_compaction_token_cap)), which defaults to the threshold. So a model can compact later than the global default only if you raise the cap.
60+
- **Retained Messages**: the percentage of the most recent messages kept verbatim, everything older being replaced by the summary (default `40`, [`CONTEXT_COMPACTION_RETENTION_PERCENTAGE`](/reference/env-configuration#context_compaction_retention_percentage)). Values outside `10` to `50` are clamped, so at least a tenth and at most half of the conversation is always kept in full.
6061
- **Context Compaction Prompt**: an optional custom summarization prompt ([`CONTEXT_COMPACTION_PROMPT_TEMPLATE`](/reference/env-configuration#context_compaction_prompt_template)); leave it empty for the built-in default. It supports `{{COMPACTED_MESSAGES}}` (the messages being summarized) and `{{RECENT_MESSAGES}}` (the messages kept in context).
6162

6263
This is the summarize-and-replace strategy (option 4 below), built in. If you need a different policy (hard caps, attachment-first trimming, per-model windows), use a filter Function instead.

0 commit comments

Comments
 (0)