Skip to content

Improve error messaging for short output token lengths #1790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Improve error messaging for short output token lengths #1790

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
46cac66
Improve error handling for short output token lengths
suryabdev Oct 2, 2025
e99af03
Remove warning from agent memory
suryabdev Oct 11, 2025
33ceea0
Add documentation
suryabdev Oct 11, 2025
0e072b8
Merge branch 'main' into fix-1783
suryabdev Oct 11, 2025
78475f2
Add unit tests
suryabdev Oct 11, 2025
4dfca23
make style
suryabdev Oct 12, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/smolagents/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ def parse_json_blob(json_blob: str) -> tuple[dict[str, str], str]:
json_data = json.loads(json_str, strict=False)
return json_data, json_blob[:first_accolade_index]
except IndexError:
raise ValueError("The model output does not contain any JSON blob.")
raise ValueError("The model output does not contain any JSON blob. Try increasing the maximum output token length.")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the tool calling agent, I've added some details to the error message. This gets fed back to the ReAct loop but only the user can edit the model parameters. Maybe we should print it at the end of the trace?

For the CodeAgent, Shall I add a similar error in parse_code_blobs?

smolagents/src/smolagents/utils.py

Line 224 in f76dee1

Your code snippet is invalid, because the regex pattern {code_block_tags[0]}(.*?){code_block_tags[1]} was not found in it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should print it at the end of the trace?

Actually thinking again, we shouldn't pass this part to the ReAct loop. It could confuse the model
Maybe print a warning message that doesn't get added to the agent's memory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we shouldn't pass this part to the ReAct loop. It could confuse the model

Modified the code to print a warning. The warning wont get added to the error and hence shouldn't get added to the agent's memory.
It won't be visible to the LLM in the next steps

smolagents/src/smolagents/agents.py

Line 596 in f042c0c

action_step.error = e

except json.JSONDecodeError as e:
place = e.pos
if json_blob[place - 1 : place + 2] == "},\n":
Expand Down