Describe the bug
When using a GGUF thinking model in chat-instruct mode, using the "Google Gemma 4 Canonical Chat Template", prior thinking content appears to remain in subsequent prompts even though 'preserve thinking' is unchecked.
The UI correctly displays generated reasoning inside a collapsed Thought box, so the output is apparently being recognized as thinking for presentation purposes. But the context sizes seemed wrong, and when I added some extra logging I found the thinking from previous messages gets put into subsequent prompts.
On the other hand the prior thinking is correctly removed when using a Qwen 3.5 gguf model (see right-behaviour-qwen.txt ). Presumably some difference in the templating or the reasoning markers is to blame? The gemma template (see gemma-gguf-embedded-template.txt ) defines preserve_thinking and contains a strip_thinking() macro - but perhaps that's not applied when prior dialogue appears as text inside the outer user message produced by chat-instruct composition?
Observed behaviour
When discarding thinking I would expect the context size to follow an upward-sloping sawtooth pattern, rising by the length of the output tokens but dropping between messages by the length of the thinking tokens. But what I see is the context constantly rising. wrong-behaviour-gemma.txt shows this in the various 'prompt size' and 'output generated' log lines.
Is there an existing issue for this?
Reproduction
- Load a model that uses the "Google Gemma 4 Canonical Chat Template"
- Enable thinking, but disable 'preserve thinking' while in the 'chat-instruct' mode.
- Have a multi-turn conversation, keeping track of the context size
- You will notice the context growing faster than expected, by about the number of thinking tokens.
Screenshot
Logs
See attached wrong-behaviour-gemma.txt
System Info
nvidia on linux, more details in the log
Thanks for taking the time to read my bug report!
Describe the bug
When using a GGUF thinking model in
chat-instructmode, using the "Google Gemma 4 Canonical Chat Template", prior thinking content appears to remain in subsequent prompts even though 'preserve thinking' is unchecked.The UI correctly displays generated reasoning inside a collapsed Thought box, so the output is apparently being recognized as thinking for presentation purposes. But the context sizes seemed wrong, and when I added some extra logging I found the thinking from previous messages gets put into subsequent prompts.
On the other hand the prior thinking is correctly removed when using a Qwen 3.5 gguf model (see right-behaviour-qwen.txt ). Presumably some difference in the templating or the reasoning markers is to blame? The gemma template (see gemma-gguf-embedded-template.txt ) defines
preserve_thinkingand contains astrip_thinking()macro - but perhaps that's not applied when prior dialogue appears as text inside the outer user message produced bychat-instructcomposition?Observed behaviour
When discarding thinking I would expect the context size to follow an upward-sloping sawtooth pattern, rising by the length of the output tokens but dropping between messages by the length of the thinking tokens. But what I see is the context constantly rising. wrong-behaviour-gemma.txt shows this in the various 'prompt size' and 'output generated' log lines.
Is there an existing issue for this?
Reproduction
Screenshot
Logs
See attached wrong-behaviour-gemma.txt
System Info
nvidia on linux, more details in the log
Thanks for taking the time to read my bug report!