[Bugfix] Fix Responses tool JSON retry

[CorgiBoyG]

Purpose

Fixes #55530.

ParsableContext currently handles malformed built-in tool arguments inconsistently. The browser and container paths try to build a Harmony error message from a ResponseFunctionToolCall, which raises AttributeError, while the code-interpreter path lets the original JSONDecodeError escape. The local demo Tool path also bypasses the existing retry handling.

This change validates built-in tool arguments once at the common ParsableContext.call_tool() dispatch boundary when VLLM_TOOL_JSON_ERROR_AUTOMATIC_RETRY is enabled. Invalid JSON is returned to the model as a ResponseFunctionToolCallOutputItem with the original call_id; no tool session is invoked. When automatic retry is disabled, the existing exception behavior is unchanged.

Test coverage

The regression tests exercise the public call_tool() path for code interpreter, browser, and container tools across both local Tool and MCP ClientSession branches. They also verify that:

• malformed arguments fail before dispatch;
• retry output preserves the originating call_id;
• skipped tools are not added to cleanup state;
• disabling retry preserves the existing JSONDecodeError;
• valid arguments still reach each ClientSession exactly once with unchanged parameters.

Validation

VLLM_TARGET_DEVICE=cpu .venv/bin/python -m pytest \
  tests/entrypoints/openai/responses/test_parsable_context_unit.py \
  tests/entrypoints/openai/responses/test_responses_utils.py \
  tests/entrypoints/openai/responses/test_protocol.py \
  tests/entrypoints/openai/responses/test_serving_responses.py -q

84 passed, 1 xfailed

The new retry tests were also run against the unmodified implementation:

6 failed

The failures were the expected JSONDecodeError and AttributeError paths described in #55530.

pre-commit run ruff-check --files <changed files>
Passed

pre-commit run ruff-format --files <changed files>
Passed

pre-commit run mypy-3.12 --files <changed files> --hook-stage manual
Passed

git diff --check
Passed

This is a control-flow fix and does not affect model output quality, model accuracy, kernels, or scheduling, so no model evaluation is applicable.

Duplicate check

No open issue or PR was found for the ParsableContext malformed-JSON retry failure. PR #47537 addresses successful built-in tool output call_id reuse and does not cover this error path. PR #47112 addresses max_tool_calls enforcement and does not overlap this change.

AI assistance was used to inspect the call path, reproduce the failure, prepare the patch, and run local validation.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 3823092d-b0d1-4eed-a8ef-e96de096d985

📥 Commits

Reviewing files that changed from the base of the PR and between 144e79c and 6407e25.

📒 Files selected for processing (2)

• tests/entrypoints/openai/responses/test_parsable_context_unit.py
• vllm/entrypoints/openai/responses/context.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Summary

Summary by CodeRabbit

• Bug Fixes
  • Improved handling of JSON arguments for tool calls.
  • Invalid tool-call arguments now provide a retry response when automatic retry is enabled.
  • When automatic retry is disabled, invalid arguments raise a parsing error while still recording the tool call.
  • Updated tool-call processing to consistently use the current response item format.

Walkthrough

ParsableContext now validates ResponseFunctionToolCall arguments centrally, returns typed retry items for invalid JSON when enabled, and preserves exception behavior when retry is disabled. Tests cover all three built-in tool paths.

Changes

ParsableContext tool JSON handling

Layer / File(s)	Summary
Centralized JSON validation and dispatch vllm/entrypoints/openai/responses/context.py	Tool handlers now use ResponseFunctionToolCall. call_tool validates arguments before dispatch and returns ResponseFunctionToolCallOutputItem errors for automatic retry.
JSON handling test coverage tests/entrypoints/openai/responses/test_parsable_context_unit.py	Parameterized tests cover valid dispatch, retry output for invalid JSON, and JSONDecodeError behavior when retry is disabled.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 6407e

Malformed built-in tool arguments now return a retryable response without dispatching the tool, while valid calls and retry-disabled behavior remain covered. The change is ready to merge.

Sequence Diagram(s)

sequenceDiagram
  participant OpenAIServingResponses
  participant ParsableContext
  participant json.loads
  participant ToolSession
  OpenAIServingResponses->>ParsableContext: call_tool(ResponseFunctionToolCall)
  ParsableContext->>json.loads: validate arguments
  alt valid JSON
    json.loads-->>ParsableContext: parsed arguments
    ParsableContext->>ToolSession: dispatch tool call
    ToolSession-->>ParsableContext: tool output
  else invalid JSON with automatic retry
    json.loads-->>ParsableContext: JSONDecodeError
    ParsableContext-->>OpenAIServingResponses: ResponseFunctionToolCallOutputItem
  end

Loading

Suggested reviewers: noooop, yzong-rh

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 23.53% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the bugfix for Responses tool JSON retry handling.
Description check	✅ Passed	The description directly explains the malformed JSON retry fix, affected tool paths, expected behavior, tests, and validation results.
Linked Issues check	✅ Passed	The changes address issue [#55530] by validating arguments at the common dispatch boundary, returning retry output with the original call_id, preventing dispatch for malformed JSON, and preserving JSO…
Out of Scope Changes check	✅ Passed	The implementation and regression tests remain focused on malformed Responses tool JSON handling and automatic retry behavior described in [#55530].

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀