Batch API

[lauhon]

Some AI providers, offer Batch APIs to favor reduced costs over performance.

It would be cool to have a simple Interface (bool param somewhere) on the Agent component

Anthropic
OpenAI

related AI-SDK Issue

[ianmacartney]

Sounds useful - given the current use of the AI SDK for LLM generation, I think the two most likely avenues are:

1. Wait for the AI SDK to have a batch API and support that
2. Better translation from Agent message types into OpenAI / Anthropic message types to make it easy to batch call those with their native APIs.