Describe the bug
When a chat consists of a single large user message that (nearly) fills the entire
context window, generation gets truncated by the llama.cpp server as expected.
However, clicking Continue afterwards crashes with:
jinja2.exceptions.UndefinedError: list object has no element 0
It appears that the message-level truncation in generate_chat_prompt removes the
only message in the history (since it alone exceeds the available context), leaving
an empty messages list. The chat template then fails when accessing messages[0],
so no prompt is built at all. As a result, Continue produces no output (or garbage
like "...").
Older versions of the webui truncated within the text (dropping tokens from the
front), so Continue still worked in this scenario. Version 4.9 fails instead.
Is there an existing issue for this?
Reproduction
- Load a GGUF model via llama.cpp with
ctx-size 32768
- Start a new chat and paste a single message of roughly the context size
(e.g. a large source file, ~32k tokens)
- Generation runs, output stops at
max_new_tokens, server log shows truncated = 1
- Click Continue
- Traceback appears in the console, no continuation is generated
Screenshot
Expected behavior
Continue should either truncate within the oversized message (as older versions did)
or fail gracefully with a user-facing warning ("message exceeds context size"),
instead of crashing during template rendering.
Logs
3.02.831.626 I slot print_timing: id 0 | task 0 | prompt eval time = 16774.42 ms / 32256 tokens
3.02.831.630 I slot print_timing: id 0 | task 0 | eval time = 121226.70 ms / 512 tokens
3.02.832.867 I slot release: id 0 | task 0 | stop processing: n_tokens = 32767, truncated = 1
20:59:33-234249 INFO Output generated in 138.05 seconds (3.72 tokens/s, 514 tokens, context 32256, seed 1343974747)
Traceback (most recent call last):
...
File "modules\chat.py", line 1237, in chatbot_wrapper
prompt = generate_chat_prompt(text, state, **kwargs)
File "modules\chat.py", line 738, in generate_chat_prompt
prompt = make_prompt(messages)
File "modules\chat.py", line 598, in make_prompt
prompt = renderer(
File "<template>", line 13, in top-level template code
File "jinja2\sandbox.py", line 319, in getattr
value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: list object has no element 0
(Full traceback available on request.)
System Info
- Version: 4.9 (full installation, one-click installer)
- OS: Windows
- Backend: llama.cpp (GGUF), ctx-size 32768
- Also reproduced with ctx-size 16384 (same traceback at the smaller limit)
Describe the bug
When a chat consists of a single large user message that (nearly) fills the entire
context window, generation gets truncated by the llama.cpp server as expected.
However, clicking Continue afterwards crashes with:
jinja2.exceptions.UndefinedError: list object has no element 0
It appears that the message-level truncation in
generate_chat_promptremoves theonly message in the history (since it alone exceeds the available context), leaving
an empty
messageslist. The chat template then fails when accessingmessages[0],so no prompt is built at all. As a result, Continue produces no output (or garbage
like "...").
Older versions of the webui truncated within the text (dropping tokens from the
front), so Continue still worked in this scenario. Version 4.9 fails instead.
Is there an existing issue for this?
Reproduction
ctx-size 32768(e.g. a large source file, ~32k tokens)
max_new_tokens, server log showstruncated = 1Screenshot
Expected behavior
Continue should either truncate within the oversized message (as older versions did)
or fail gracefully with a user-facing warning ("message exceeds context size"),
instead of crashing during template rendering.
Logs
System Info