Skip to content

Gemini structured output fails with includeThoughts: true #798

@fobtastic

Description

@fobtastic

Description

When using Gemini's structured output with includeThoughts: true, the response fails to parse because Prism's Structured.php handler doesn't filter out thought parts like the Text.php handler does.

Steps to Reproduce

use Prism\Prism\Facades\Prism;
use Prism\Prism\Enums\Provider;
use Prism\Prism\Schema\ObjectSchema;
use Prism\Prism\Schema\StringSchema;

$schema = new ObjectSchema(
    name: 'test',
    properties: [
        new StringSchema('result', 'A result')
    ]
);

$response = Prism::structured()
    ->using(Provider::Gemini, 'gemini-2.0-flash-exp')
    ->withSystemPrompt('Extract information')
    ->withPrompt('Test input')
    ->withSchema($schema)
    ->withProviderOptions([
        'thinkingConfig' => [
            'thinkingLevel' => 'low',
            'includeThoughts' => true,
        ],
    ])
    ->asStructured();

Expected Behavior

The structured output should be extracted from the actual content part (the non-thought part), similar to how Text.php handles it.

Actual Behavior

Error: Structured object could not be decoded. Received: **Thinking Through...

The response contains the thinking text instead of the JSON because Prism reads from parts[0].text, which contains the thinking when includeThoughts: true.

Root Cause

When includeThoughts: true, Gemini returns multiple parts:

  • parts[0]: { thought: true, text: "thinking content..." }
  • parts[1]: { thought: false, text: "{\"result\":\"...\"}" }

Currently in src/Providers/Gemini/Handlers/Structured.php, the code always reads from parts[0]:

$responseMessage = new AssistantMessage(
    data_get($data, 'candidates.0.content.parts.0.text') ?? '',
    // ...
);

And in addStep():

structured: $isStructuredStep ? $this->extractStructuredData(data_get($data, 'candidates.0.content.parts.0.text') ?? '') : [],

Proposed Fix

The Structured.php handler should filter parts similar to how Text.php does:

protected function extractTextContent(array $data): string
{
    $parts = data_get($data, 'candidates.0.content.parts', []);
    $textParts = [];

    foreach ($parts as $part) {
        // Only include text from parts that are NOT thoughts
        if (isset($part['text']) && (!isset($part['thought']) || $part['thought'] === false)) {
            $textParts[] = $part['text'];
        }
    }

    return implode('', $textParts);
}

Additionally, it would be valuable to extract and expose the thought content separately (similar to how Text.php extracts thoughtSummaries), so developers can access the reasoning for debugging purposes.

Workaround

Set includeThoughts: false (or omit it entirely) to avoid the issue. The model will still use thinking internally, but the thought content won't be included in the response.

Environment

  • Prism version: 0.99.2
  • Provider: Gemini
  • Model: gemini-3.0-flash-preview (Gemini Flash 3 with thinking support)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions