Skip to content

withToolChoice() is ignored when using Prism::structured() with Anthropic and custom tools #821

@CamilleScholtz

Description

@CamilleScholtz

Description

When using Prism::structured() with Anthropic, custom tools, and use_tool_calling: true, (also happens with false and the structured output beta header), the withToolChoice() setting is completely ignored. The model can skip calling custom tools entirely and produce structured output directly, even when ToolChoice::Any or a specific tool name is specified.

Steps to Reproduce

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Enums\ToolChoice;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;
use Prism\Prism\Tool;

$schema = new ObjectSchema(
    name: 'result',
    description: 'Result with verified data',
    properties: [
        new StringSchema('answer', 'The answer'),
    ],
    requiredFields: ['answer']
);

$searchTool = Tool::as('search')
    ->for('Search for information')
    ->withStringParameter('query', 'Search query')
    ->using(fn (string $query): string => "Results for: {$query}");

$response = Prism::structured()
    ->using(Provider::Anthropic, 'claude-sonnet-4-5-20250514')
    ->withSchema($schema)
    ->withTools([$searchTool])
    ->withToolChoice(ToolChoice::Any) // This is ignored!
    ->withProviderOptions(['use_tool_calling' => true])
    ->withMaxSteps(5)
    ->withPrompt('Search for the capital of France and tell me the answer')
    ->asStructured();

// Expected: Model calls 'search' tool first, then returns structured output
// Actual: Model returns structured output directly without calling 'search'

Expected Behavior

The model should be forced to call at least one of the custom tools (search) before returning the structured output, respecting the ToolChoice::Any setting.

Actual Behavior

The model skips all custom tool calls and returns the structured output directly in a single step with finishReason: Stop.

Root Cause Analysis

The issue is in ToolStructuredStrategy::mutatePayload():

// src/Providers/Anthropic/Handlers/StructuredStrategies/ToolStructuredStrategy.php:56-58
if ($this->request->providerOptions('thinking.enabled') !== true && $this->request->tools() === []) {
    $payload['tool_choice'] = ['type' => 'tool', 'name' => self::STRUCTURED_OUTPUT_TOOL_NAME];
}

The tool_choice is only set when there are no custom tools. When custom tools are present, this condition is false, so no tool_choice is added to the payload.

Additionally, Structured::buildHttpRequestPayload() never includes the user's toolChoice in the base payload - it's not even read from the request.

Environment

  • Prism version: 0.99.10
  • PHP version: 8.5.1
  • Laravel version: 12

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions