Skip to content

fix(data): return content when llama3/mistral tool extractors find no calls#10592

Open
he-yufeng wants to merge 1 commit into
hiyouga:mainfrom
he-yufeng:fix/llama3-tool-extractor-empty
Open

fix(data): return content when llama3/mistral tool extractors find no calls#10592
he-yufeng wants to merge 1 commit into
hiyouga:mainfrom
he-yufeng:fix/llama3-tool-extractor-empty

Conversation

@he-yufeng

Copy link
Copy Markdown

What does this PR do?

tool_extractor returns a str (the original content) when the assistant message contains no function calls, and a list[FunctionCall] otherwise. Llama3ToolUtils.tool_extractor and MistralToolUtils.tool_extractor returned the list comprehension directly, so a payload that parses to an empty list (e.g. "[]") produced [] instead of the content string:

Llama3ToolUtils.tool_extractor("[]")   # -> []   (should be "[]")
MistralToolUtils.tool_extractor("[]")  # -> []   (should be "[]")

The Qwen3.5, Seed, and LFM2 extractors already guard this with return results if results else content (Seed via #10408). This PR applies the same guard to the llama3 and mistral extractors so they stay consistent with the documented str | list[FunctionCall] contract.

Tests

Added test_llama3_tool_extractor_empty_returns_content and test_mistral_tool_extractor_empty_returns_content in tests/data/test_formatter.py, asserting ToolFormatter(tool_format=...).extract("[]") == "[]". Both fail before the change (they return []) and pass after.

PYTHONPATH=src python -m pytest tests/data/test_formatter.py   # 36 passed
ruff check . && ruff format --check .                          # clean on the changed files

… calls

Llama3ToolUtils.tool_extractor and MistralToolUtils.tool_extractor returned the
raw list comprehension, so a payload that parses to an empty list (e.g. "[]")
yielded [] instead of the original content string. tool_extractor returns str
when there are no function calls; the Qwen3.5/Seed/LFM2 extractors already guard
this with 'return results if results else content' (Seed via hiyouga#10408). Apply the
same guard to the llama3 and mistral extractors.

Add tests asserting both return the content string for an empty tool array.

Signed-off-by: Yufeng He <40085740+he-yufeng@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request modifies the tool extractors in src/llamafactory/data/tool_utils.py to return the original content string instead of an empty list when no tool calls are extracted, and adds corresponding unit tests in tests/data/test_formatter.py. The review feedback correctly identifies potential TypeError vulnerabilities if the parsed tools are not dictionaries, as the current try-except block only catches KeyError. It is recommended to validate the structure of the tools before processing them.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +444 to +446
results = [
FunctionCall(tool["name"], json.dumps(tool["parameters"], ensure_ascii=False)) for tool in tools
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If tools contains elements that are not dictionaries (e.g., if the model outputs a list of integers or strings, or nested lists), accessing tool["name"] or tool["parameters"] will raise a TypeError (e.g., 'int' object is not subscriptable). Since the try-except block only catches KeyError, this will cause an unhandled exception and crash the extraction process.

To make this more robust and adhere to defensive programming practices, we should validate that each tool in tools is a dictionary and contains the required keys before attempting to construct the FunctionCall objects.

Suggested change
results = [
FunctionCall(tool["name"], json.dumps(tool["parameters"], ensure_ascii=False)) for tool in tools
]
if not all(isinstance(tool, dict) and "name" in tool and "parameters" in tool for tool in tools):
return content
results = [
FunctionCall(tool["name"], json.dumps(tool["parameters"], ensure_ascii=False)) for tool in tools
]

tools = [tools] if not isinstance(tools, list) else tools
try:
return [FunctionCall(tool["name"], json.dumps(tool["arguments"], ensure_ascii=False)) for tool in tools]
results = [FunctionCall(tool["name"], json.dumps(tool["arguments"], ensure_ascii=False)) for tool in tools]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Similarly, if tools contains elements that are not dictionaries, accessing tool["name"] or tool["arguments"] will raise a TypeError which is not caught by the except KeyError: block.

We should validate that each tool in tools is a dictionary and contains the required keys before attempting to construct the FunctionCall objects.

Suggested change
results = [FunctionCall(tool["name"], json.dumps(tool["arguments"], ensure_ascii=False)) for tool in tools]
if not all(isinstance(tool, dict) and "name" in tool and "arguments" in tool for tool in tools):
return content
results = [FunctionCall(tool["name"], json.dumps(tool["arguments"], ensure_ascii=False)) for tool in tools]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant