You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lead the essentials context section with Context Compaction
The context management section still said Open WebUI ships no way to handle a full context window and sent everyone straight to writing a filter Function. Context Compaction has covered the common case for a while now, so a reader following that page was being told to build something that already exists.
It now leads with compaction: what it does, that an administrator has to turn it on first and where, the threshold to set, and the two limits worth knowing before relying on it, that crossing the threshold does not guarantee it ran and that the token figure is an estimate where the provider reported none. Filter Functions stay as the answer for any other policy. The troubleshooting page also gains the Retained Messages setting, which the admin panel has but its list was missing.
Copy file name to clipboardExpand all lines: docs/getting-started/essentials.mdx
+16-5Lines changed: 16 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -131,18 +131,29 @@ After enough back-and-forth you will eventually see:
131
131
This error comes from your model provider, not from Open WebUI. Every time you send a message, the *entire* conversation (system prompt, all previous turns, attached files, tool call results, and your new message) is sent as the "prompt." When the sum exceeds the model's context window, the provider rejects the request.
132
132
:::
133
133
134
-
Open WebUI intentionally does not ship a built-in trimmer, because:
134
+
### The built-in answer: Context Compaction
135
135
136
-
- Every model uses a **different tokenizer** (GPT, Claude, Gemini, GLM, Qwen all differ).
137
-
- Every model has a **different context window** (8k to 1M+).
138
-
- Every deployment wants a **different policy** (trim by tokens, by turns, by message count, drop attachments first, summarize older messages, etc.).
136
+
Open WebUI ships **Context Compaction** for exactly this. Once a chat's estimated context passes a token threshold, a model summarizes the older messages, and the conversation continues from that summary plus the most recent messages, so a long chat keeps going instead of hitting the wall. It runs server-side and changes only what is sent to the model on that turn. Your chat is still stored and displayed in full, and nothing is deleted.
139
137
140
-
There is no single correct answer. The supported approach is to install a **filter Function** that trims the conversation on your terms.
138
+
It is **off by default**. An administrator turns it on in **Settings > Admin > Experience > Interface**, where the token threshold (default `80000`), the model that writes the summaries, the share of recent messages kept verbatim and an optional custom summary prompt are all configurable. Set the threshold below your model's real context window.
139
+
140
+
Two things to know before you lean on it:
141
+
142
+
- Crossing the threshold does not guarantee it ran. The cut has to land on a user message, so a chat that is one enormous exchange rather than many turns is sent uncompacted. That happens silently and is retried on the next turn, so compaction is a relief valve rather than a hard ceiling.
143
+
- The token figure is an estimate wherever your provider did not report real usage, so read it as a gauge rather than an exact count.
144
+
145
+
With it enabled, `/compact` in the chat input compacts the conversation right away instead of waiting for the threshold.
146
+
147
+
### When you need a different policy
148
+
149
+
Compaction is the summarize-and-replace strategy. For a hard cap on chat length, attachments dropped before dialogue, tool results trimmed first or a different window per model, install a **filter Function** instead. Open WebUI ships no blind trimmer, because tokenizers, context windows and the right policy all differ per model and per deployment.
141
150
142
151
:::tip Quick fix
143
152
Community filters for most common policies already exist and can be installed with one click. If none fits, the code is short enough to copy and adapt. See the full guide including a minimal "newest N turns" filter: [Troubleshooting: Context Window / Prompt Too Long](/troubleshooting/context-window).
144
153
:::
145
154
155
+
More detail: [Context Compaction](/troubleshooting/context-window#the-built-in-option-context-compaction) covers the checkpointing, the fallbacks when the summarizer fails, and the `context_usage` API for showing how close a chat is to compacting.
Copy file name to clipboardExpand all lines: docs/troubleshooting/context-window.mdx
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -57,6 +57,7 @@ It is **disabled by default**. An administrator enables and tunes it in **Settin
57
57
-**Context Compaction**: the on/off toggle ([`ENABLE_CONTEXT_COMPACTION`](/reference/env-configuration#enable_context_compaction)).
58
58
-**Context Compaction Model**: the model that writes the summaries ([`CONTEXT_COMPACTION_MODEL`](/reference/env-configuration#context_compaction_model)). Leave it on **Current Model** to follow the task model as before.
59
59
-**Token Threshold**: the estimated context size, in tokens, above which older messages are compacted (default `80000`, [`CONTEXT_COMPACTION_TOKEN_THRESHOLD`](/reference/env-configuration#context_compaction_token_threshold)). Set it below your model's real window. This is the global default: an individual model can set its own **Context Compaction Threshold** (`compact_token_threshold`) in its advanced parameters, either lower or higher, bounded by the **Token Cap** ([`CONTEXT_COMPACTION_TOKEN_CAP`](/reference/env-configuration#context_compaction_token_cap)), which defaults to the threshold. So a model can compact later than the global default only if you raise the cap.
60
+
-**Retained Messages**: the percentage of the most recent messages kept verbatim, everything older being replaced by the summary (default `40`, [`CONTEXT_COMPACTION_RETENTION_PERCENTAGE`](/reference/env-configuration#context_compaction_retention_percentage)). Values outside `10` to `50` are clamped, so at least a tenth and at most half of the conversation is always kept in full.
60
61
-**Context Compaction Prompt**: an optional custom summarization prompt ([`CONTEXT_COMPACTION_PROMPT_TEMPLATE`](/reference/env-configuration#context_compaction_prompt_template)); leave it empty for the built-in default. It supports `{{COMPACTED_MESSAGES}}` (the messages being summarized) and `{{RECENT_MESSAGES}}` (the messages kept in context).
61
62
62
63
This is the summarize-and-replace strategy (option 4 below), built in. If you need a different policy (hard caps, attachment-first trimming, per-model windows), use a filter Function instead.
0 commit comments