Improve StructuredOutput options

[gnat42]

Hello,

I'm not sure if the issue is how I'm using the system or if there needs to be some improvements when trying to use structured output. We're trying to get some objects out from anthropic's API.

What it supports (in terms of json_schema) seems to be somewhat fickle, because of this I don't think we can pass a object or class name to the response_format. When we do, Anthropic complains. Because of this we did the following:

        if ($agentMessageConfig->getOutputSchema()) {
            $options[PlatformSubscriber::RESPONSE_FORMAT] = [
                'json_schema' => [
                    'schema' => json_decode($agentMessageConfig->getOutputSchema(), true, 512, JSON_THROW_ON_ERROR),
                ],
            ];
        }
...
        return $this->agent->call($messages, $options);

This works fairly well, except then we get an array returned instead of the object. This happens because src/StructuredOutput/PlatformSubscriber.php's logic is

 $responseFormat = $options[self::RESPONSE_FORMAT];

        if (\is_object($responseFormat)) {
            $this->objectToPopulate = $responseFormat;
            $className = $responseFormat::class;
        } elseif (\is_string($responseFormat) && class_exists($responseFormat)) {
            $this->objectToPopulate = null;
            $className = $responseFormat;
        } else {
            return;
        }

Because we've provided the schema as an array already, no $className is set. So later on in the processResult() method we have

public function processResult(ResultEvent $event): void
{
        $options = $event->getOptions();

        if (!isset($options[self::RESPONSE_FORMAT])) {
            return;
        }

        $deferred = $event->getDeferredResult();
        $converter = new ResultConverter(
            $deferred->getResultConverter(),
            $this->serializer,
            $this->outputType ?? null,
            $this->objectToPopulate
        );

        $event->setDeferredResult(new DeferredResult($converter, $deferred->getRawResult(), $options));

        // Reset object to populate for next invocation
        $this->objectToPopulate = null;
}

Where the self::RESPONSE_FORMAT is set, but outputType is null, and so we just get back an array because the ResultConverter is instantiated without an outputType.

I'm not sure what the correct solution is.

I could create an event subscriber with the correct priority to replace the auto generated schema with the desired one but keeping the class. However, I'm not sure if that's the desired solution. I'm happy to code things up and provide a PR but need a bit of guidance on how you expect something like this to work.

[gnat42]

hmm, I just discovered/noticed one more difference, everything here is using the symfony serializer, and the objects we have configured are using jms.

So I'm curious as to other options for requesting an object and handling the conversion with other tools.