max_tokens in requests somehow set to 128 #5269
Replies: 2 comments
-
The IssueIn agent mode, you might notice that responses are sometimes abruptly cut off at exactly 128 tokens, even if you set a higher Why it happensOdysseus has an internal intent classifier that reads your prompt before sending it to the agent loop. If a prompt is very short or doesn't explicitly look like it requires complex tool usage, the classifier labels it as a "low-signal turn" (usually meant for casual conversational chatter). To reply faster and save compute resources for these casual chats, Odysseus routes low-signal turns through a "fast-path". However, this fast-path had a hardcoded limit that forced max_tokens=min(max_tokens or 128, 128)The FixTo resolve this, you need to remove the hardcoded 128 limit so the fast-path respects your configured token limits. Open src/agent_loop.py. max_tokens=min(max_tokens or 128, 128),To this: max_tokens=max_tokens,Restart the Odysseus backend, and your agent will now be able to return full-length responses for short prompts! |
Beta Was this translation helpful? Give feedback.
-
|
I hope it helps !! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hello, guys!
Please help me understand what I’m doing wrong.
I choose agent mode and ask it to do something, but it always cuts the answer off at 128 tokens.
I’m running llama-server, and in verbose mode I see this request from Odysseus:
"74.25.704.164 D srv log_server_r: request: {"model":"Qwen3.6-35B-A3-Uncensored-HauhauCS-Aggressive-Q2_K_P.gguf","messages":[{"role":"user","content":"execute command pwd"}],"temperature":1.0,"stream":true,"stream_options":{"include_usage":true},"max_tokens":128,"session_id":"42a349f5-3517-48d1-b2a2-46f2d1909367","cache_prompt":true}"
I don’t understand where this "max_tokens":128 is coming from in request or how to remove it.
Beta Was this translation helpful? Give feedback.
All reactions