Bionic context compaction dead-ends on homepage demo task with local 65K Qwen3.8 model

[matusbielik]

Environment

• App: Bionic
• OS: Debian 13
• GPU: NVIDIA RTX 4090, 24 GB
• NVIDIA driver: 595.58.03
• Backend: llama.cpp Linux x86_64 CUDA 12, version 2.31.2
• Model: lmstudio-community/Qwen3.8-27B-GGUF
• Quant: Qwen3.8-27B-Q4_K_M.gguf
• Model context: 65,536 tokens
• GPU offload: 65 / maximum
• CPU thread pool: 8
• Max concurrent predictions: 1
• Flash Attention: enabled
• KV cache: GPU offload enabled, Unified KV Cache enabled
• Speculative decoding: MTP, max draft tokens 3
• Reasoning budget: 4,096 tokens

What happened

I used the same office-plant/PDF prompt shown in the Bionic homepage demo:

Research large office plants suited to north-facing light and a 3 m ceiling.
Create a PDF with one candidate per page, including images and a table
of light and watering requirements.

Bionic appeared to perform context compaction during the task. However,
eventually it failed with:

The selected model was loaded with a context length that is too small
for this request. Increase the context length of the model, then reload
and try again.

At that point, the Bionic session was unrecoverable: even a short follow-up
could not be submitted, so Bionic could not run another compaction or
produce a continuation summary.

There is no UI setting for Bionic's compaction trigger, context reserve,
or emergency overflow recovery.

Expected behaviour

When a local model has a 65,536-token context, Bionic should compact early
enough to leave space for:

• the compaction request itself,
• system/tool prompts,
• the current user request, and
• a bounded compaction response.

If compaction fails or is triggered too late, Bionic should fall back to a
recoverable strategy, such as deterministic history truncation plus a
short recovery summary, rather than dead-ending the session.

Relevant evidence

• The homepage appears to use the same task with Kimi K3 High and displays
  around 231.7K context, while the local Qwen session is limited to 65.5K.
• The local server logs show Bionic allowing very large individual outputs,
  including one generation of 12,414 tokens.
• The active runtime context reached at least 48,372 tokens before the
  session became unusable.
• Reducing the reasoning budget from 8,192 to 4,096 did not prevent the
  problem.

Note

I was able to reproduce the problem multiple times with various model settings - I was actually never able to produce a pdf like in the advertised example

Attachments

2026-09-01.1.log