Skip to content

"Continue" crashes with jinja2 UndefinedError when a single message exceeds the context size #7629

Description

@SoehnelS

Describe the bug

When a chat consists of a single large user message that (nearly) fills the entire
context window, generation gets truncated by the llama.cpp server as expected.
However, clicking Continue afterwards crashes with:

jinja2.exceptions.UndefinedError: list object has no element 0

It appears that the message-level truncation in generate_chat_prompt removes the
only message in the history (since it alone exceeds the available context), leaving
an empty messages list. The chat template then fails when accessing messages[0],
so no prompt is built at all. As a result, Continue produces no output (or garbage
like "...").

Older versions of the webui truncated within the text (dropping tokens from the
front), so Continue still worked in this scenario. Version 4.9 fails instead.

Is there an existing issue for this?

  • I have searched the existing issues

Reproduction

  1. Load a GGUF model via llama.cpp with ctx-size 32768
  2. Start a new chat and paste a single message of roughly the context size
    (e.g. a large source file, ~32k tokens)
  3. Generation runs, output stops at max_new_tokens, server log shows truncated = 1
  4. Click Continue
  5. Traceback appears in the console, no continuation is generated

Screenshot

Expected behavior
Continue should either truncate within the oversized message (as older versions did)
or fail gracefully with a user-facing warning ("message exceeds context size"),
instead of crashing during template rendering.

Logs

3.02.831.626 I slot print_timing: id  0 | task 0 | prompt eval time = 16774.42 ms / 32256 tokens
3.02.831.630 I slot print_timing: id  0 | task 0 |        eval time = 121226.70 ms / 512 tokens
3.02.832.867 I slot      release: id  0 | task 0 | stop processing: n_tokens = 32767, truncated = 1
20:59:33-234249 INFO     Output generated in 138.05 seconds (3.72 tokens/s, 514 tokens, context 32256, seed 1343974747)
Traceback (most recent call last):
  ...
  File "modules\chat.py", line 1237, in chatbot_wrapper
    prompt = generate_chat_prompt(text, state, **kwargs)
  File "modules\chat.py", line 738, in generate_chat_prompt
    prompt = make_prompt(messages)
  File "modules\chat.py", line 598, in make_prompt
    prompt = renderer(
  File "<template>", line 13, in top-level template code
  File "jinja2\sandbox.py", line 319, in getattr
    value = getattr(obj, attribute)
jinja2.exceptions.UndefinedError: list object has no element 0

(Full traceback available on request.)

System Info

- Version: 4.9 (full installation, one-click installer)
- OS: Windows
- Backend: llama.cpp (GGUF), ctx-size 32768
- Also reproduced with ctx-size 16384 (same traceback at the smaller limit)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions