Skip to content

[Agent] Tool-argument resolution failures are not surfaced to the LLM in a correctable form #2137

@sjahns

Description

@sjahns

Summary

When ToolCallArgumentResolver::resolveArguments() fails — because the LLM supplied an argument that cannot be denormalized (invalid backed-enum value, malformed object/discriminator structure, missing constructor argument) or omitted a mandatory parameter — the resulting exception is caught by Toolbox::execute() in its generic \Throwable branch and wrapped as ToolExecutionException::executionFailed(). Its getToolCallResult() returns only:

An error occurred while executing tool "X".

FaultTolerantToolbox relays exactly that string to the LLM. The actual cause (which argument was invalid, what the valid values are, or which parameter was missing) survives only in the exception message (i.e. in logs), never in the agent-facing result.

These failures are caused by the LLM's own tool-call input and are exactly the kind of feedback it could act on, but it receives none — so it cannot self-correct and typically repeats the same invalid call.

Background — the failure flow

1. The resolver throws. A missing mandatory parameter throws ToolException:

https://github.com/symfony/ai/blob/v0.9.0/src/agent/src/Toolbox/ToolCallArgumentResolver.php#L78-L79

if (!$reflectionParameter->isOptional()) {
    throw new ToolException(\sprintf('Parameter "%s" is mandatory for tool "%s".', $name, $toolCall->getName()));
}

An undeserializable value makes the serializer throw inside denormalize():

https://github.com/symfony/ai/blob/v0.9.0/src/agent/src/Toolbox/ToolCallArgumentResolver.php#L104-L106

if ($this->denormalizer->supportsDenormalization($value, $parameterType, 'json')) {
    $value = $this->denormalizer->denormalize($value, $parameterType, 'json');
}

(e.g. NotNormalizableValueException for an invalid backed-enum value, MissingConstructorArgumentsException for a malformed object argument).

2. Toolbox::execute() wraps it generically. resolveArguments() runs inside the try block; the catch arms are:

https://github.com/symfony/ai/blob/v0.9.0/src/agent/src/Toolbox/Toolbox.php#L114-L120

} catch (ToolExecutionExceptionInterface $e) {
    throw $e;
} catch (\Throwable $e) {
    $this->logger->warning(\sprintf('Failed to execute tool "%s".', $toolCall->getName()), ['exception' => $e]);

    throw ToolExecutionException::executionFailed($toolCall, $e);
}

Neither failure implements ToolExecutionExceptionInterfaceToolException implements ExceptionInterface (not the execution interface), and the serializer exceptions are plain Symfony Serializer exceptions — so both bypass the first arm and hit the generic \Throwable arm.

3. The cause is dropped from the agent-facing result. executionFailed() keeps the cause in the exception message, but getToolCallResult() returns a generic string:

https://github.com/symfony/ai/blob/v0.9.0/src/agent/src/Toolbox/Exception/ToolExecutionException.php

public static function executionFailed(ToolCall $toolCall, \Throwable $previous): self
{
    $exception = new self(\sprintf('Execution of tool "%s" failed with error: %s', $toolCall->getName(), $previous->getMessage()), previous: $previous);
    $exception->toolCall = $toolCall;

    return $exception;
}

public function getToolCallResult(): string
{
    return \sprintf('An error occurred while executing tool "%s".', $this->toolCall->getName());
}

4. FaultTolerantToolbox relays only that generic string to the LLM:

https://github.com/symfony/ai/blob/v0.9.0/src/agent/src/Toolbox/FaultTolerantToolbox.php#L36-L41

public function execute(ToolCall $toolCall): ToolResult
{
    try {
        return $this->innerToolbox->execute($toolCall);
    } catch (ToolExecutionExceptionInterface $e) {
        return new ToolResult($toolCall, $e->getToolCallResult());

Observed behavior

  • Invalid backed-enum value: the LLM calls a tool with {"color": "not_a_color"} for a Color $color parameter. The serializer throws The data must belong to a backed enumeration of type .... The LLM is told only: An error occurred while executing tool "X".
  • Missing mandatory parameter: the LLM omits a required parameter. The resolver throws Parameter "x" is mandatory for tool "Y".. The LLM is told only: An error occurred while executing tool "X".

In both cases the underlying message names exactly what to fix, but the LLM never sees it.

Minimal reproducer

The deserialization failure itself, showing the exception that reaches Toolbox::execute() is a plain serializer exception (not a ToolExecutionExceptionInterface), so it is wrapped into the generic result by the chain above:

<?php

use Symfony\AI\Agent\Toolbox\ToolCallArgumentResolver;
use Symfony\AI\Platform\Result\ToolCall;
use Symfony\AI\Platform\Tool\ExecutionReference;
use Symfony\AI\Platform\Tool\Tool;

enum Color: string
{
    case Red = 'red';
    case Green = 'green';
    case Blue = 'blue';
}

final class SomeTool
{
    public function __invoke(Color $color): array
    {
        return [];
    }
}

$resolver = new ToolCallArgumentResolver();
$tool     = new Tool(new ExecutionReference(SomeTool::class), 'some_tool', 'desc');
$toolCall = new ToolCall('tc-1', 'some_tool', ['color' => 'not_a_color']);

$resolver->resolveArguments($tool, $toolCall);
// NotNormalizableValueException (a plain Serializer exception, not a ToolExecutionExceptionInterface)
//   -> Toolbox::execute() generic \Throwable arm -> ToolExecutionException::executionFailed()
//   -> FaultTolerantToolbox relays getToolCallResult(): 'An error occurred while executing tool "some_tool".'

Note

Could argument-resolution failures be treated as a recoverable, agent-correctable category whose cause reaches the LLM — e.g. by surfacing the underlying ToolException / serializer message in the tool result for this class of error? The messages already name the missing parameter or the invalid value (for backed enums, the expected type), which is precisely the feedback the model needs to retry correctly. (If the generic getToolCallResult() is intentional to avoid leaking internal exception details to the model, note that these particular errors originate from the model's own input, not from internal tool state.)

Environment

  • symfony/ai-agent: v0.9.0
  • symfony/ai-platform: v0.9.0
  • symfony/ai-bundle: v0.9.0
  • Symfony: 7.4
  • PHP: 8.4

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    AgentIssues & PRs about the AI Agent component

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions