diff --git a/LICENSES/MIT.txt b/LICENSES/MIT.txt new file mode 100644 index 0000000..3399cd9 --- /dev/null +++ b/LICENSES/MIT.txt @@ -0,0 +1,11 @@ +SPDX-License-Identifier: MIT + +MIT License + +Copyright (c) + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/NOTICE b/NOTICE index 655a4dd..ca511b8 100644 --- a/NOTICE +++ b/NOTICE @@ -15,6 +15,8 @@ Axl uses highlight.js 11.12.0 as an external dependency for syntax highlighting. Axl uses grok-mermaid 0.2.2 as an external dependency for Unicode Mermaid rendering. It is distributed under its own Apache-2.0 license. +Axl's generated model catalog includes factual metadata derived from models.dev, retrieved from https://models.dev/api.json. Models.dev is Copyright 2025 models.dev contributors and is distributed under the MIT license. + Axl uses `@deepseek-ai/node-addon-landlock-run` 0.1.1 from DeepSeek Harness. Its JavaScript package includes an MIT notice, and its platform launcher packages are distributed under BSD-3-Clause. Those notices remain in the installed dependency diff --git a/README.md b/README.md index 90e0376..92a325e 100644 --- a/README.md +++ b/README.md @@ -33,7 +33,7 @@ Axl is not yet a hosted service, remote collaboration product, browser applicati | Durability | Append-only canonical JSONL, operation IDs, crash-safe mutation journal, restart reconciliation, and deterministic replay | | Multi-client behavior | Independent attachments, paged snapshots, acknowledged cursors, presence, reconnect recovery, and shared deterministic projection | | Automation | One-shot text and canonical JSONL output with `axl print` and `axl json`, plus native daemon RPC with `axl rpc` | -| Model interaction | Azure OpenAI model catalog, model selection, thinking levels, streaming text and reasoning, tool calls, steering, and follow-ups | +| Model interaction | Provider-grouped text-model catalog, provider-qualified model selection, authentication management, usage and costs, streaming text and reasoning, tool calls, steering, and follow-ups | | Built-in tools | `read`, `write`, `edit`, `bash`, `web_fetch`, and `web_search` | | Extensions | Public extension API, prompt templates, Agent Skills, and MCP 2025-11-25 over stdio and Streamable HTTP | | Workspace review | Bounded file listing and reads, Git status, structured diffs, and daemon-owned last-turn checkpoints | @@ -180,13 +180,17 @@ pnpm run install:cli ## Quick start -Configure Azure OpenAI and start a session: +Inspect providers, authenticate one, choose its model, and start a session: ```bash -axl login +axl providers +axl models openai +axl login openai api_key axl ``` +The `provider` and `model` startup options select a canonical pair for a new session. In the TUI, use `/model` to choose a grouped provider and model pair. + Common entry points: ```bash @@ -206,17 +210,22 @@ axl rpc # bridge JSONL RPC over stdin and stdout The CLI connects to the matching local daemon and starts one in the background when necessary. Native, OCI, and unsafe placements use separate state and are labeled in the resume picker. -## Provider authentication +## Model providers + +Use `axl providers` for explicit authentication and catalog status, `axl models` for grouped text models, `axl login` and `axl logout` for stored authentication, and `axl refresh` for explicit catalog refresh. These commands report actionable authentication, entitlement, region, catalog, model, and configuration failures. Add named local or hosted endpoints in `~/.axl/models.json`; see the [native configuration example](docs/provider-support/provider-reference.md#user-configured-endpoints). + +Inside the TUI, `/model` selects a provider-qualified model. `/providers`, `/login`, `/logout`, and `/refresh` expose the same daemon-owned operations. Escape cancels an active provider operation. The editor reports last-turn and cumulative token usage and USD cost when available. -Provider secrets never pass through the TUI or SDK projection. +Provider secrets never pass through daemon RPC or SDK projection. -1. The CLI collects login input and writes the credential store with restrictive permissions. -2. `packages/ai` implements provider-specific credential parsing, verification, model metadata, and request behavior. -3. `packages/runtime` resolves the selected provider inside the daemon process. -4. The TUI receives only a provider-neutral login dialog definition from the CLI process host. -5. Canonical events, SDK cursors, and client projections never contain live credentials. +1. The daemon owns provider and session operations. +2. `packages/ai` owns provider-specific credentials, authentication, model metadata, API dialects, and request behavior. +3. The trusted CLI process-host adapter renders provider prompts and collects answers inside the daemon process. +4. Login RPC carries only the provider ID and login method. +5. Authorization launch is restricted to HTTPS URLs without embedded credentials. +6. Canonical events, SDK cursors, catalogs, and client projections never contain credential values, OAuth codes, or prompt answers. -Azure OpenAI is the built-in provider today. Provider-specific behavior does not belong in the kernel, protocol, SDK, or presentation clients. +Provider listing is offline and side-effect free. Authentication status and catalog refresh are separate explicit operations. API dialect is model metadata, not user-selectable configuration. See the [provider setup and compatibility reference](docs/provider-support/provider-reference.md) for every provider, environment variable, endpoint, region, authentication method, catalog type, custom-endpoint boundary, limitation, and opt-in smoke procedure. Provider authority and client responsibilities follow [`docs/architecture/client-boundaries.md`](docs/architecture/client-boundaries.md). ## Session profiles diff --git a/REUSE.toml b/REUSE.toml index 436ea47..abfdf19 100644 --- a/REUSE.toml +++ b/REUSE.toml @@ -133,6 +133,19 @@ SPDX-FileCopyrightText = [ ] SPDX-License-Identifier = "Apache-2.0" +[[annotations]] +path = ["packages/ai/catalog/sources/models-dev/**"] +SPDX-FileCopyrightText = "2025 models.dev contributors" +SPDX-License-Identifier = "MIT" + +[[annotations]] +path = [ + "packages/ai/catalog/semantic-baseline.json", + "packages/ai/catalog/sources/ant-ling/**", +] +SPDX-FileCopyrightText = "2026 Kaushik Kumar" +SPDX-License-Identifier = "Apache-2.0" + # Build output is disposable and inherits its source license. [[annotations]] path = ["packages/*/dist/**", "packages/extensions/*/dist/**"] diff --git a/biome.json b/biome.json index 54602b8..c211fda 100644 --- a/biome.json +++ b/biome.json @@ -1,7 +1,13 @@ { "$schema": "https://biomejs.dev/schemas/2.3.5/schema.json", "files": { - "includes": ["**/*.{ts,json}", "!**/{dist,node_modules}", "!.release"] + "includes": [ + "**/*.{ts,json}", + "!**/{dist,node_modules}", + "!.release", + "!packages/ai/src/catalog.generated.ts", + "!packages/ai/src/catalog.generated" + ] }, "formatter": { "enabled": true, diff --git a/docs/architecture/web-protocol.md b/docs/architecture/web-protocol.md index aeae762..fbd48c5 100644 --- a/docs/architecture/web-protocol.md +++ b/docs/architecture/web-protocol.md @@ -13,13 +13,13 @@ This document specifies typed RPC, negotiation, errors, package ownership, and t ## Current baseline -Wire version 11 uses newline-delimited JSON over a Unix socket. It includes typed request and result envelopes, initialization, capability negotiation, structured errors, idempotency keys, subscription identities, paged snapshots, acknowledged opaque cursors, presence, daemon security reporting, direct shell events, transient activity, session-bound blobs, workspace review, session profiles, web-tool selection, manual compaction, steering, follow-ups, and canonical model-retry attempts. +Wire version 12 uses newline-delimited JSON over a Unix socket. It includes typed request and result envelopes, initialization, capability negotiation, structured errors, idempotency keys, subscription identities, paged snapshots, acknowledged opaque cursors, presence, daemon security reporting, direct shell events, transient activity, session-bound blobs, workspace review, session profiles, web-tool selection, manual compaction, steering, follow-ups, canonical model-retry attempts, provider management, and model-request configuration. -The TUI consumes these contracts through `packages/sdk`. Version 11 adds daemon-owned model request settings plus canonical effective-request events. Host-control version 1 remains separate from session wire negotiation and is available only to trusted process hosts. +The TUI consumes these contracts through `packages/sdk`. The two former branch tips both used version 11 for incompatible additions: provider management on the feature branch and daemon-owned request settings on `main`. Version 12 combines both surfaces. Host-control version 1 remains separate from session wire negotiation and is available only to trusted process hosts. ## Versioning -The current wire version is 11. Version 8 introduced typed envelopes, initialization, errors, retry metadata, subscriptions, cursors, acknowledgements, and presence. Version 9 adds the canonical `model.retry_scheduled` event. Version 10 adds `daemon_stopping` as a pre-RPC and universal RPC error. Version 11 adds `config.request` and `model.request_configured` events and request settings in session create and configure RPCs. Compatible capability additions that do not alter accepted wire data do not require a bump. Pre-1.0 clients require an exact wire-version match. +The current wire version is 12. Version 8 introduced typed envelopes, initialization, errors, retry metadata, subscriptions, cursors, acknowledgements, and presence. Version 9 adds the canonical `model.retry_scheduled` event. Version 10 adds `daemon_stopping` as a pre-RPC and universal RPC error. The two incompatible version-11 development surfaces are superseded. Version 12 combines provider-management RPCs with `config.request`, `model.request_configured`, and request settings in session create and configure RPCs. Compatible capability additions that do not alter accepted wire data do not require a bump. Pre-1.0 clients require an exact wire-version match. The daemon sends `hello` first: diff --git a/docs/model-provider-protocol-compatibility.md b/docs/model-provider-protocol-compatibility.md new file mode 100644 index 0000000..520780e --- /dev/null +++ b/docs/model-provider-protocol-compatibility.md @@ -0,0 +1,35 @@ + + + +# Model provider protocol compatibility + +## Scope + +The issue 10 provider contract extends the in-process model stream shared by `packages/protocol`, `packages/ai`, and `packages/kernel`. Provider-management RPCs changed the daemon wire surface. Current `main` independently added model-request configuration under wire version 11. The combined surface therefore uses wire version 12. The persisted JSONL event format remains version 1. + +## Additive stream behavior + +Existing providers and consumers remain valid. `ModelProvider.refreshModels(): Promise` remains the legacy additive refresh hook. Context-aware providers use `refreshModelCatalog(context)`, which returns `ModelCatalogRefreshResult`. Legacy string dialect identifiers and boolean compatibility maps remain assignable; built-in codecs still require and validate their typed dialect records. + +- Text and thinking deltas may omit `contentIndex`. +- A complete `tool_call` remains the authoritative instruction consumed by the kernel. +- `tool_call_start` and `tool_call_delta` provide optional progress without replacing the complete call. +- Completion, error, and abort remain the only terminal variants, and exactly one terminal event is still required. +- Response attribution, partial-content status, retry guidance, and diagnostics are optional terminal metadata. +- `replay_metadata` is optional nonterminal metadata for one positioned thinking, text, or tool-call block. It does not replace visible content or a complete `tool_call`. Signed thinking may additionally set `redacted: true` when its opaque signature must be replayed as a provider redacted-thinking block rather than ordinary signed thinking. + +New codecs should provide stable `contentIndex` values whenever the upstream protocol can interleave text, thinking, and tool blocks. Consumers that do not render incremental tool arguments may ignore progress events and wait for `tool_call`. Consumers that retain provider replay metadata must bind it to the identified content block and exact provider, dialect, and model. + +## Trust boundary + +`parseModelStreamEvent` validates provider events before normalized streams enter the kernel. The safe diagnostic contract accepts only a code, message, and severity. It intentionally has no arbitrary details, headers, request bodies, stack traces, or credential fields. + +Response metadata may contain provider identity, requested and routed model identity, a response ID, native stop detail, and latency. A `replay_metadata` event may contain only exact issuing provider, dialect, and model identity, a content position, an optional tool-call ID, the narrow opaque signature or continuation fields needed for same-model replay, and an optional redacted-thinking marker. That marker is valid only for a thinking target with a signature. Replay metadata cannot carry headers, credentials, arbitrary provider objects, or diagnostics. Empty replay metadata, malformed identities, and mismatched targets fail validation. + +The OpenAI Responses codec emits these events for completed reasoning, text, and tool-call items. Azure OpenAI uses the same stream grammar and records the canonical `azure-openai-responses` provider and dialect identity. OpenAI Codex records the `openai-codex-responses` dialect and replays the complete prepared history for stateless SSE requests. It does not infer `previous_response_id` from history because the reviewed Codex continuation is scoped to a proven live connection, account, request baseline, and response prefix. Anthropic Messages emits signed-thinking replay metadata and marks opaque redacted-thinking signatures so the next request reconstructs the correct Anthropic block type. Google Generative AI may attach a thought signature to text, thinking, or tool-call parts. The signature does not classify a part as thinking, only the native `thought` marker does that. Session model-port adapters retain Google signatures on their matching block without inventing continuation state when no continuation identifier was emitted. Google Vertex AI uses the same event conversion but binds replay metadata to the distinct `google-vertex` dialect and provider identity. Bedrock Converse Stream emits signed or encrypted reasoning replay metadata under the `bedrock-converse-stream` dialect and retains provider event positions across interleaved text, thinking, and tool calls. Mistral Conversations replays native visible thinking without opaque replay metadata, preserves positions across interleaved thinking, text, and fragmented tool calls, and reports the native finish reason with requested and routed model identity. Gateway messages carries prepared context to a dynamically routed backend, preserves text, thinking, and tool signatures only for the exact gateway model, accepts gateway-reported usage and cost rather than applying one guessed route price, and reports requested and routed model identity plus the native stop reason. Session model-port adapters retain replay events in memory and attach them to matching assistant history before the next prepared dispatch. Retention is limited to the live port instance and exact issuing model identity. Persisted JSONL events and daemon wire versions remain unchanged, so restart and history reconstruction intentionally do not restore this metadata. + +## Model and request metadata + +The `packages/ai` additions are optional for existing callers. API dialects, compatibility controls, endpoint policy, cache policy, availability, tiered prices, sampling, cache preferences, timeout, and bounded retry controls are interpreted by provider adapters. The kernel remains provider independent. + +Native image generation is an optional capability on `ModelProvider`. Image bytes travel through blob reader and writer callbacks, while results contain content-addressed blob references rather than inline bytes. The OpenRouter image codec keeps this operation outside the canonical text stream: it validates and reads content-addressed reference images, stores each generated image through `writeBlob`, and returns provider, requested model, optional routed model and response identity, revised prompt, usage, cost, and blob references. Cancellation and codec or provider failures reject the image operation with typed, redacted errors rather than inventing text-stream terminal events. diff --git a/docs/provider-support/deterministic-verification.md b/docs/provider-support/deterministic-verification.md new file mode 100644 index 0000000..a784366 --- /dev/null +++ b/docs/provider-support/deterministic-verification.md @@ -0,0 +1,83 @@ + + + +# Model provider deterministic verification matrix + +## Scope + +This record maps Step 12 requirements to executable local tests. Tests use injected fetch functions, in-memory stores, temporary directories, and checked-in manifests. They do not contact live providers or use live credentials. + +Equivalent existing tests are retained as the requirement evidence. New tests are added only where the audit finds a missing integration boundary. + +## Provider registration and dialect fixtures + +- All 41 built-in provider identities: `builtin-providers.test.ts`, `registers exactly all 41 planned provider identities`. +- Listing has no credential or network side effects: `builtin-providers.test.ts`, `registers exactly all 41 planned provider identities`; `catalog.test.ts`, `static catalog access performs no network or credential work`. +- Catalog-selected dialect and endpoint policy: `builtin-providers.test.ts`, `preserves catalog selected dialects and exact endpoint policies`. +- OpenAI Chat and Responses transports: `remaining-providers.test.ts`, `dispatches every newly active static provider through its declared dialect endpoint`. +- Azure OpenAI Responses transport: `azure-openai.test.ts`, `streams from Azure with api-key header, versioned URL, and mapped deployment`. +- OpenAI Codex Responses transport: `remaining-providers.test.ts`, `dispatches Codex, Gateway, and image dialects through deterministic transports`. +- Anthropic Messages transport: `remaining-providers.test.ts`, `dispatches every newly active static provider through its declared dialect endpoint`. +- Google Generative AI and Vertex transports: `remaining-providers.test.ts`, `dispatches every newly active static provider through its declared dialect endpoint`. +- Bedrock Converse Stream transport: `aws-auth.test.ts`, `Bedrock signs every dispatch through the AWS default credential chain`. +- Mistral Conversations transport: `remaining-providers.test.ts`, `dispatches every newly active static provider through its declared dialect endpoint`. +- Gateway messages transport: `remaining-providers.test.ts`, `dispatches Codex, Gateway, and image dialects through deterministic transports`. +- OpenRouter image transport: `remaining-providers.test.ts`, `dispatches Codex, Gateway, and image dialects through deterministic transports`. +- Full codec request and stream fixtures: `openai-chat.test.ts`, `openai-responses.test.ts`, `azure-openai.test.ts`, `openai-codex-responses.test.ts`, `anthropic-messages.test.ts`, `google-generative-ai.test.ts`, `google-vertex.test.ts`, `bedrock-converse-stream.test.ts`, `mistral-conversations.test.ts`, `gateway-messages.test.ts`, and `openrouter-images.test.ts`. + +## Authentication and catalog lifecycle + +- Stored credential precedence: `auth.test.ts`, `a stored api key owns the provider over the environment`. +- Environment, file, ambient, and keyless order: `auth.test.ts`, `ambient authentication uses fixed environment, file, ambient, and keyless precedence`. +- Failed stored credentials never fall through: `auth.test.ts`, `stored credential failure never falls back through ambient precedence`. +- OAuth refresh serialization and failure: `auth.test.ts`, `expiring oauth refreshes exactly once across concurrent resolutions`; `a failed refresh surfaces refresh_failed with no silent fallback`. +- Login, cancellation, supersession, and logout races: `auth.test.ts`, lifecycle tests from `interactive authorization reports UI-neutral lifecycle states` through `logout wins a race with an in-flight refresh`. +- Provider OAuth protocols: `subscription-auth.test.ts`. +- Azure, Vertex, and Bedrock ambient credentials: `cloud-auth.test.ts` and `aws-auth.test.ts`. +- Credential persistence and metadata-only listing: `credentials.test.ts`. +- Generated catalog coverage and reproducibility: `catalog.test.ts`, `generated catalog covers every planned provider identity`; `catalog artifact deterministically matches local manifests and overlays`. +- Catalog validation, provenance, and regional isolation: `catalog.test.ts`. +- Reasoning-map shape, value types, and at least one supported level: `catalog.test.ts`, `reasoning maps require an object with valid values and a supported level`. +- All bundled Azure models across all seven Axl thinking levels: `azure-openai.test.ts`, `every generated Azure model encodes its declared reasoning map`. +- Native models.json rejects malformed reasoning, output limits, and missing or mismatched compatibility records: `models-config.test.ts`. +- Atomic provider-scoped catalog persistence: `catalog-store.test.ts`. +- Offline restoration before network work: `registry.test.ts`, `restores a persisted dynamic catalog before network refresh`. +- Failed and malformed refresh retention: `registry.test.ts`, `failed and malformed refreshes retain the previous valid catalog`. +- Cancelled and superseded refresh races: `registry.test.ts`, `cancelled and superseded refreshes cannot replace the last-known-good catalog`. +- Corrupt provider isolation: `registry.test.ts`, `dynamic refresh isolates corrupt persisted providers from healthy providers`. + +## Integration boundaries and redaction + +- Mixed-dialect model selection: `registry.test.ts`, `dispatches mixed dialect models through their owning provider`. +- Mixed-dialect built-in transport selection: `remaining-providers.test.ts`, `dispatches every newly active static provider through its declared dialect endpoint`, including OpenAI, OpenCode, and OpenCode Go. +- Keyless configured Chat endpoint and validated custom headers: `remaining-providers.test.ts`, `dispatches a keyless configured endpoint with only validated custom headers`. +- API-key configured Responses endpoint: `remaining-providers.test.ts`, `dispatches a configured Responses endpoint with explicit API key authentication`. +- Shared diagnostics and authentication redaction: `model-contract.test.ts`, `provider diagnostics redact known secrets and remain bounded`; `auth.test.ts`, `authentication states and diagnostics expose no credential values`. +- Transport and codec redaction: provider failure tests in `deepseek-provider.test.ts`, `azure-openai.test.ts`, `openai-chat.test.ts`, `anthropic-messages.test.ts`, `google-generative-ai.test.ts`, `bedrock-converse-stream.test.ts`, `mistral-conversations.test.ts`, `gateway-messages.test.ts`, and `openrouter-images.test.ts`. +- Same-model continuation retention: `request-preparation.test.ts`, `retains replay metadata only for its exact issuing model`; `provider-port.test.ts`, `retains replay metadata in assistant history for the next in-process turn`. +- Cross-provider continuation sanitization: `provider-port.test.ts`, `strips foreign continuation state when a session changes providers`. +- Foreign opaque reasoning rejection: `request-preparation.test.ts`, `rejects foreign redacted reasoning instead of dropping opaque content`. +- Dynamic and custom endpoint policy, restored-origin dispatch checks, fail-closed rows, Anthropic environment headers, and image timeout: `remaining-providers.test.ts`. +- SSE line, event, frame, and total limits: `sse.test.ts`. +- AWS frame and split-prelude limits: `aws-event-stream.test.ts`. +- Bounded buffered JSON and linear URL normalization: `transport-safety.test.ts`. +- Real pinned-DNS HTTP transport with Node's single-address and address-array callbacks: `transport-safety.test.ts`, `real transport honors both Node DNS lookup callback shapes`. +- Vertex and Bedrock SDK cancellation: `cloud-auth.test.ts` and `aws-auth.test.ts`. +- Persistence-commit supersession and legacy provider source compatibility: `registry.test.ts`. +- Azure interactive login: `cloud-auth.test.ts` and the runtime provider inventory assertion in `local-runtime.test.ts`. + +## Invariants + +The matrix preserves canonical `{ providerId, modelId }` selection. API dialect stays model metadata. Provider listing remains side-effect free. Authentication and credential values remain inside provider-owned trusted processes. No test introduces a compatibility fallback or live provider dependency. + +## Separate opt-in Azure smoke, 2026-09-07 + +The user explicitly authorized Azure testing. The built CLI and built runtime ran against Azure in a disposable workspace with Bubblewrap enforced. The existing Azure API key was read into an isolated in-memory store. The user's credential and settings files were hash-checked before and after and remained unchanged. No other provider inference was tested. + +- Public catalog refresh initially failed because the pinned DNS callback ignored Node's `all` option. The shared transport was fixed without disabling address validation or DNS pinning. Azure metadata refresh then succeeded with 67 models. +- Two minimal inference calls were made with `gpt-5.6-luna` and `low` reasoning. The first CLI call exited successfully, but the temporary smoke reporter used incorrect canonical event names. The corrected reporter verified the second call returned exactly `OK`, stopped normally, and recorded requested/effective thinking as `low` without clamping. +- The verified call reported 195 input tokens, 5 output tokens, 0 reasoning tokens, and catalog-derived cost of $0.000045. Zero reasoning tokens on this trivial prompt does not prove that reasoning is disabled; the requested level was accepted. +- No tool calls occurred. Credential values were absent from stdout, stderr, and the canonical session record. The isolated daemon and temporary session were cleaned up. +- `pnpm check` passed with 810 tests passing and 8 existing platform/environment skips. The bundled catalog retained its reviewed 1,102-model baseline, including 66 Azure models. Offline Azure reasoning encoding covered 462 model/level combinations. + +This smoke verifies one configured Azure model at `low`, not every Azure deployment or advertised reasoning level. It is not part of the automated test suite. diff --git a/docs/provider-support/implementation-notes.md b/docs/provider-support/implementation-notes.md new file mode 100644 index 0000000..30fdee8 --- /dev/null +++ b/docs/provider-support/implementation-notes.md @@ -0,0 +1,81 @@ + + + +# Provider implementation notes + +## Scope + +This document records durable provenance and compatibility decisions for Axl's built-in model providers. User configuration, provider identities, authentication methods, endpoints, and known limitations are documented in the [provider reference](provider-reference.md). Catalog provenance and regeneration are documented in [`packages/ai/catalog/README.md`](../../packages/ai/catalog/README.md). + +Provider behavior belongs in `packages/ai`. The protocol and kernel remain provider independent. Catalog and provider registration are offline and side-effect free. Dynamic discovery, authentication, and network requests occur only through explicit operations. + +## Behavioral reference + +Axl's codecs were implemented independently after reviewing Pi at commit `92d8e2d17d4f357788381c49ce2cdb3f4ed1f21c`. No Pi source or generated catalog is copied into Axl. The reference is used to identify interoperability behavior that public provider specifications do not fully describe. + +Static model metadata was independently normalized from the checked-in models.dev and Ant Ling source manifests. The models.dev source revision is `5c600a037417cf778ee6eb3ea2ce0f17abc12130`. The manifests record retrieval details and checksums. + +## Native API sources + +The primary public specifications reviewed for native codecs and provider composition are: + +- OpenAI Chat and Responses: +- Azure OpenAI Responses: +- Azure endpoint migration: +- Anthropic Messages: +- Google Generative AI: +- Vertex AI locations: +- Vertex Express Mode: +- Bedrock Converse Stream: +- Mistral Conversations: +- OpenRouter models: +- OpenRouter images: +- Cloudflare AI Gateway authentication: +- Cloudflare OpenAI-compatible chat: +- GitHub Copilot authentication: +- OpenCode Zen and Go: and + +OpenAI does not publish the ChatGPT Codex subscription backend as a stable public API. Codex support is therefore pinned to the reviewed behavior and fails explicitly when that behavior cannot be represented safely. + +## Authentication sources + +Cloud and subscription authentication was reviewed against: + +- OpenAI Codex authentication: +- Anthropic authentication: +- GitHub OAuth device flow: +- OpenRouter OAuth: +- Azure managed identity: +- Google Application Default Credentials: +- Vertex authentication: +- AWS standardized credential providers: +- AWS Signature Version 4: + +The official Azure Identity, Google Auth Library, AWS credential-provider, and Smithy signing packages own cloud credential acquisition and signing. Credentials stay inside provider-owned stores, SDK credential objects, or signing closures. They are never included in provider metadata, canonical events, prompts, diagnostics, or public SDK projections. + +## Compatibility decisions + +- A session selects `{ providerId, modelId }`; the model catalog selects the API dialect. +- Requests pass through provider-neutral preparation before provider dispatch. +- Opaque reasoning signatures and continuation identifiers are retained only for the exact issuing provider, dialect, and model. +- Required strict schemas and unsupported controls fail instead of silently degrading. +- Provider endpoints are validated before credentials or prompts are attached. User-configured HTTP endpoints are limited to explicit loopback development addresses. +- Dynamic catalogs are bounded, validated, persisted atomically, and revalidated before dispatch. Failed, cancelled, corrupt, or superseded refreshes retain the last-known-good generation. +- HTTP and SDK operations propagate cancellation. Retries are bounded and stop once streamed output is exposed. +- Listing providers and static models does not read credentials, perform network requests, or start background work. + +## Verification + +Routine tests use deterministic fake transports, credential stores, SDK clients, and local fixtures. They do not call live providers. The requirement-to-test mapping is maintained in [deterministic verification](deterministic-verification.md). + + +## Pi comparison for PR 385 + +Read-only behavioral reference: Pi revision `6c87d9a026677b601e8278030dcf1ad97fe0bd86`. + +- Pi's user-authored `~/.pi/agent/models.json` configures custom providers and overrides. It is distinct from generated provider catalogs and the disposable `models-store.json` cache. Its generator fetches models.dev and selected provider catalogs and applies maintained corrections. Axl keeps independent source metadata and policy. +- Pi's explicit command is `pi update --models`. It forces refreshable configured provider catalogs; it is not `pi --model refresh`. At this reference revision, `openaiProvider()` supplies static models without `refreshModels`, so that command does not regenerate every built-in provider. Axl's requested models.dev refresh is broader and remains explicit rather than running in the background. +- Pi's `/login` offers account/API-key selection, filtered provider selection, and inline prompts. Axl now follows that interaction sequence with its existing picker, dialog, and trusted process-host boundary. No Pi implementation was copied. +- Axl's native models.json intentionally excludes executable credential commands and built-in provider overrides. Named custom providers can use different endpoints without redirecting an existing built-in credential. +- Authentication status checks in this PR can resolve ambient cloud credentials. Listing inventory alone remains offline, but `/providers`, the login provider picker, and configured-only refresh are explicit checks, not guaranteed offline metadata reads. +- Catalog refresh is not evidence of live inference compatibility. The provider inventory and deterministic codec tests do not establish that all subscription backends, entitlements, or model IDs work against live services. Broad claims of complete live provider support remain premature without explicit provider-by-provider smoke tests. diff --git a/docs/provider-support/provider-reference.md b/docs/provider-support/provider-reference.md new file mode 100644 index 0000000..92e6125 --- /dev/null +++ b/docs/provider-support/provider-reference.md @@ -0,0 +1,183 @@ + + + +# Model provider setup and compatibility reference + +## Scope and selection boundary + +Axl registers 41 provider identities, including the `custom` library integration. A session always selects the canonical pair `{ providerId, modelId }`. The model catalog selects the API dialect. Users cannot select a dialect independently or use it as a provider identity. + +Use `axl providers [provider-id]` to inspect authentication and catalog status, `axl models [provider-id]` to list text models, `axl login [api_key|oauth]` to store credentials, `axl logout ` to remove them, and `axl refresh [provider-id]` to refresh configured catalogs explicitly. Listing provider metadata does not read credentials, contact providers, or refresh catalogs. + +Stored credentials take precedence over environment, file, ambient, and keyless sources. A stored credential that fails does not fall through to another source. Interactive authentication runs inside the trusted daemon process-host adapter. Credential values, OAuth codes, tokens, and prompt answers do not cross daemon RPC. + +## Built-in provider matrix + +Endpoint paths shown below are the effective request base or full request endpoint. Static catalogs are checked in and available offline. Dynamic catalogs restore a validated last-known-good snapshot when one exists, then change only through explicit refresh. + +| Provider ID | Authentication and environment | Endpoint and region | Catalog and API dialect | Important limitation | +| --- | --- | --- | --- | --- | +| `openai` | API key, `OPENAI_API_KEY` | `https://api.openai.com/v1` | Static, model-selected Chat or Responses | No automatic dialect fallback | +| `azure-openai-responses` | API key, `AZURE_OPENAI_API_KEY`, or Microsoft Entra | Configured Azure resource or base URL, Responses path | Static, Azure Responses | Requires base URL or resource name and deployment alignment | +| `openai-codex` | ChatGPT subscription OAuth only | `https://chatgpt.com/backend-api/codex/responses` | Static, Codex Responses | Undocumented backend is pinned to the reviewed behavior; SSE sends full stateless history | +| `anthropic` | API key, `ANTHROPIC_API_KEY` or `ANTHROPIC_OAUTH_TOKEN`; subscription OAuth | `https://api.anthropic.com/v1/messages` | Static, Anthropic Messages | Signed thinking is retained only in the live process | +| `google` | API key, `GEMINI_API_KEY` then `GOOGLE_API_KEY` | Google Generative Language `v1beta` streaming endpoint | Static, Google Generative AI | Thought signatures are retained only in the live process | +| `google-vertex` | Express API key, `GOOGLE_CLOUD_API_KEY`; service account; or ADC | Vertex Express, global, multi-region, regional, or configured collection endpoint | Static, Google Vertex | ADC and service-account modes require a location; project must be configured or discoverable | +| `amazon-bedrock` | Bedrock bearer token, named AWS profile, or AWS default credential chain | Bedrock Runtime in the selected region; inference-profile ARNs can select routing region | Static, Bedrock Converse Stream | Region is required; SigV4 acquisition or signing failure never falls back to unsigned dispatch | +| `github-copilot` | GitHub or Copilot token, `COPILOT_GITHUB_TOKEN`; GitHub device OAuth | Token-selected individual, business, or enterprise endpoint | Dynamic entitlement catalog, catalog-selected dialect | Requires explicit refresh before first use unless a valid cache exists | +| `xai` | API key, `XAI_API_KEY`; subscription device OAuth | `https://api.x.ai/v1` | Static, OpenAI Chat | OAuth depends on the reviewed Grok CLI subscription flow | +| `deepseek` | API key, `DEEPSEEK_API_KEY` | `https://api.deepseek.com/chat/completions` | Static, OpenAI Chat | Chat reasoning-detail emission remains limited as described below | +| `mistral` | API key, `MISTRAL_API_KEY` | `https://api.mistral.ai/v1/conversations` | Static, Mistral Conversations | Grammar tools and opaque continuation metadata are unsupported | +| `groq` | API key, `GROQ_API_KEY` | `https://api.groq.com/openai/v1/chat/completions` | Static, OpenAI Chat | Compatibility is model metadata, not inferred from provider name | +| `cerebras` | API key, `CEREBRAS_API_KEY` | `https://api.cerebras.ai/v1/chat/completions` | Static, OpenAI Chat | Catalog declares no prompt-cache support | +| `nvidia` | API key, `NVIDIA_API_KEY` | `https://integrate.api.nvidia.com/v1/chat/completions` | Static, OpenAI Chat | Compatibility is model metadata, not inferred from provider name | +| `openrouter` | API key, `OPENROUTER_API_KEY`; browser PKCE OAuth | `https://openrouter.ai/api/v1` for models, Chat, and Images | Dynamic text and image catalogs; Chat and native Images | Explicit refresh is required without a cache; no first-party image command is exposed yet | +| `vercel-ai-gateway` | API key, `AI_GATEWAY_API_KEY` | `https://ai-gateway.vercel.sh/v1/chat/completions` | Static, OpenAI Chat | Vercel OIDC is not implemented | +| `cloudflare-ai-gateway` | API token, `CLOUDFLARE_API_KEY`, plus account and gateway IDs | Cloudflare unified gateway `compat` endpoint | Dynamic, catalog-selected dialect | Requires explicit refresh and both account settings | +| `cloudflare-workers-ai` | API token, `CLOUDFLARE_API_KEY`, plus account ID | Cloudflare account-scoped Workers AI Chat endpoint | Static, OpenAI Chat | Account identity is separate from AI Gateway configuration | +| `fireworks` | API key, `FIREWORKS_API_KEY` | `https://api.fireworks.ai/inference/v1/chat/completions` | Static, OpenAI Chat | Axl does not silently switch to Fireworks' Anthropic-compatible surface | +| `together` | API key, `TOGETHER_API_KEY` | `https://api.together.ai/v1/chat/completions` | Static, OpenAI Chat | Uses only the reviewed OpenAI-compatible surface | +| `baseten` | API key, `BASETEN_API_KEY` | `https://inference.baseten.co/v1/chat/completions` | Static, OpenAI Chat | Compatibility is model metadata | +| `huggingface` | API token, `HF_TOKEN` | `https://router.huggingface.co/v1/chat/completions` | Static, OpenAI Chat | The catalog excludes models that do not satisfy the reviewed contract | +| `zai` | API key, `ZAI_API_KEY` | `https://api.z.ai/api/paas/v4/chat/completions`, global | Static, OpenAI Chat | Global and China credentials and catalogs are separate | +| `zai-coding-cn` | API key, `ZAI_CODING_CN_API_KEY` | `https://open.bigmodel.cn/api/coding/paas/v4/chat/completions`, China | Static, OpenAI Chat | China coding-plan identity is isolated from `zai` | +| `minimax` | API key, `MINIMAX_API_KEY` | `https://api.minimax.io/v1/chat/completions`, global | Static, OpenAI Chat | Axl selects the OpenAI-compatible surface, not the separate Anthropic surface | +| `minimax-cn` | API key, `MINIMAX_CN_API_KEY` | `https://api.minimaxi.com/v1/chat/completions`, China | Static, OpenAI Chat | Separate regional credential and catalog | +| `moonshotai` | API key, `MOONSHOT_API_KEY` | `https://api.moonshot.ai/v1/chat/completions`, global | Static, OpenAI Chat | Shares an environment variable with the China identity but not stored credentials | +| `moonshotai-cn` | API key, `MOONSHOT_API_KEY` | `https://api.moonshot.cn/v1/chat/completions`, China | Static, OpenAI Chat | Separate regional endpoint and stored credential | +| `kimi-coding` | API key, `KIMI_API_KEY`; subscription device OAuth | `https://api.kimi.com/coding/v1/chat/completions` | Static, OpenAI Chat | OAuth depends on the reviewed Kimi Code public-client flow | +| `qwen-token-plan` | API key, `QWEN_TOKEN_PLAN_API_KEY` | Alibaba Singapore token-plan endpoint | Static, OpenAI Chat | Regional identity is separate from China | +| `qwen-token-plan-individual` | API key, `QWEN_TOKEN_PLAN_API_KEY` | `https://coding-intl.dashscope.aliyuncs.com/v1/chat/completions` | Static, OpenAI Chat | Shares an environment variable with the Singapore plan but not stored credentials | +| `qwen-token-plan-cn` | API key, `QWEN_TOKEN_PLAN_CN_API_KEY` | Alibaba Beijing token-plan endpoint, China | Static, OpenAI Chat | Separate regional credential and catalog | +| `xiaomi` | API key, `XIAOMI_API_KEY` | `https://api.xiaomimimo.com/v1/chat/completions`, global API billing | Static, OpenAI Chat | Separate from all token-plan identities | +| `xiaomi-token-plan-cn` | API key, `XIAOMI_TOKEN_PLAN_CN_API_KEY` | Xiaomi token-plan endpoint, China | Static, OpenAI Chat | Separate regional credential and catalog | +| `xiaomi-token-plan-ams` | API key, `XIAOMI_TOKEN_PLAN_AMS_API_KEY` | Xiaomi token-plan endpoint, Amsterdam | Static, OpenAI Chat | Separate regional credential and catalog | +| `xiaomi-token-plan-sgp` | API key, `XIAOMI_TOKEN_PLAN_SGP_API_KEY` | Xiaomi token-plan endpoint, Singapore | Static, OpenAI Chat | Separate regional credential and catalog | +| `opencode` | API key, `OPENCODE_API_KEY` | `https://opencode.ai/zen/v1` | Static, model-selected Chat, Responses, Messages, or Google | Dialect follows the official model table; there is no compatibility fallback | +| `opencode-go` | API key, `OPENCODE_API_KEY` | `https://opencode.ai/zen/go/v1` | Static, model-selected Chat, Responses, or Messages | Shares an environment variable with Zen but not stored credentials | +| `ant-ling` | API key, `ANT_LING_API_KEY` | `https://api.ant-ling.com/v1/chat/completions` | Static, OpenAI Chat | Catalog declares no prompt-cache support | +| `radius` | API key, `RADIUS_API_KEY`; gateway browser or device OAuth | Configured gateway, default `https://radius.pi.dev`; `/v1/config` discovery and returned `/messages` base | Dynamic, Gateway messages | Public wire and OAuth contracts are not fully stable; explicit refresh is required without a cache | +| `custom` | Caller-selected API-key environment names or keyless mode | Caller-supplied HTTPS base URL, or HTTP only on an explicit loopback address | Caller-supplied models and dialect metadata | Native `models.json` adds named providers; keyless or environment-backed authentication | + +## Endpoint and regional settings + +| Setting | Provider | Meaning | +| --- | --- | --- | +| `AZURE_OPENAI_BASE_URL` | Azure OpenAI | Explicit Azure, proxy, or gateway base URL | +| `AZURE_OPENAI_RESOURCE_NAME` | Azure OpenAI | Builds `https://.openai.azure.com/openai/v1` when no base URL is set | +| `AZURE_OPENAI_API_VERSION` | Azure OpenAI | API version query value, default `v1` | +| `AZURE_OPENAI_DEPLOYMENT_NAME_MAP` | Azure OpenAI | Comma-separated `model=deployment` mappings | +| `GOOGLE_CLOUD_PROJECT` or `GCLOUD_PROJECT` | Vertex | Project for ADC or service-account routing | +| `GOOGLE_CLOUD_LOCATION` | Vertex | Required location for ADC and service-account modes; supports `global`, `us`, `eu`, and regional locations | +| `GOOGLE_APPLICATION_CREDENTIALS` | Vertex | Service-account credential file; selected before ambient ADC | +| `GOOGLE_VERTEX_BASE_URL` | Vertex | Explicit collection endpoint | +| `GOOGLE_VERTEX_API_VERSION` | Vertex | Validated API version, default `v1` | +| `AWS_REGION` or `AWS_DEFAULT_REGION` | Bedrock | Request and signing region; `AWS_REGION` wins | +| `AWS_PROFILE` | Bedrock | Named profile for the AWS credential chain | +| `GITHUB_ENTERPRISE_URL` or `GH_HOST` | GitHub Copilot | Enterprise GitHub host; the stored setting wins | +| `CLOUDFLARE_ACCOUNT_ID` | Cloudflare providers | Account used in the endpoint path | +| `CLOUDFLARE_GATEWAY_ID` | Cloudflare AI Gateway | Gateway used in the endpoint path | + +Provider settings are not credentials unless explicitly identified as a key or token. They are still validated and remain provider scoped. Azure, Vertex, Bedrock, Cloudflare, Copilot, and regional identities fail with actionable configuration errors when required settings are absent or malformed. + +## Authentication methods + +API-key providers accept provider-scoped interactive key entry and the environment variable listed in the matrix. OAuth is implemented for OpenAI Codex, Anthropic, GitHub Copilot, OpenRouter, Kimi For Coding, Radius, and xAI. Browser flows use PKCE where supported. Device flows obey expiry, cancellation, polling intervals, and `slow_down` guidance. Authorization URLs rendered by first-party clients must be validated HTTPS URLs without embedded credentials. + +Azure supports provider-scoped interactive API-key login, which stores the key with its required Azure base URL. Azure uses the official Azure Identity default credential chain when no stored key or `AZURE_OPENAI_API_KEY` is available. Vertex uses the official Google Auth Library for service accounts and ADC. Bedrock uses `AWS_BEARER_TOKEN_BEDROCK` before the official AWS default credential chain, which includes environment, SSO, web identity, shared configuration, process, container, and instance-role sources. Cloud SDK token caches and refresh remain inside those SDK credential objects. + +`axl logout ` removes only the named provider's stored credential. It does not alter environment variables, cloud tool configuration, or another regional identity. + +## Compatibility controls + +Compatibility controls are reviewed model metadata, not free-form user configuration. They determine system or developer roles, output-token field names, reasoning formats and budgets, strict or grammar tools, tool-result quirks, streamed usage, cache controls and retention, session affinity, gateway routing, sampling allowlists, safety controls, and provider-specific continuation replay. + +Request preparation rejects a control when the selected model and dialect do not declare a safe wire representation. Required strict schemas are never silently weakened. Custom sampling fields require an explicit model allowlist. Authentication-shaped headers and metadata are rejected. Opaque signatures and continuation identifiers are retained only for the exact issuing provider, dialect, and model; foreign continuation state is removed, while unsafe foreign redacted reasoning fails closed. + +Behavioral provenance and durable compatibility decisions are recorded in [`implementation-notes.md`](implementation-notes.md). The executable requirement map is in [`deterministic-verification.md`](deterministic-verification.md). + +## User-configured endpoints + +`createCustomProvider` supports caller-supplied model metadata for OpenAI Chat, OpenAI Responses, Anthropic Messages, Google Generative AI, Mistral Conversations, and Gateway messages. This covers compatible servers such as Ollama, llama.cpp, vLLM, SGLang, and LM Studio only when the caller supplies accurate model capabilities and dialect metadata. + +The base URL must use HTTPS unless it is an explicit loopback development server. Loopback, private, link-local, multicast, and local-name remote destinations are rejected. Embedded URL credentials, fragments, and endpoint queries are forbidden. Custom headers must be non-secret and pass catalog validation; authorization, cookie, proxy authorization, API-key, token, credential, password, and secret-shaped headers are forbidden. Authentication is either keyless or uses explicit caller-selected environment-variable names. A missing model list, missing base URL, unsupported dialect, unsafe header, or unsupported compatibility control fails explicitly. + +The daemon and trusted login host load named providers from `~/.axl/models.json`. Each provider owns its credential-store key. Names cannot shadow built-ins, except that `custom` may replace the empty built-in placeholder. Omit `apiKeyEnvironmentVariables` for a keyless endpoint; otherwise credentials can be supplied by those variables or stored with `axl login api_key`. + +```json +{ + "providers": { + "local": { + "displayName": "Local server", + "baseUrl": "http://127.0.0.1:11434/v1", + "models": [{ + "modelId": "qwen-local", + "displayName": "Local Qwen", + "apiDialect": "openai-chat", + "capabilities": { "toolUse": true, "structuredOutput": false, "imageInput": false }, + "reasoning": false, + "contextWindow": 32768, + "maxOutputTokens": 4096, + "compatibility": { "dialect": "openai-chat", "supportsDeveloperRole": false } + }] + } + } +} +``` + +Use the model ID and limits actually configured on your server. Each entry accepts `displayName`, `baseUrl`, a non-empty `models` array using Axl's `ModelInfo` fields, optional non-secret `headers`, and optional `apiKeyEnvironmentVariables`. Model `providerId` and endpoint are assigned from the enclosing provider. An explicit compatibility record is required. Unknown fields, unsupported dialects, unsafe endpoints, secret-shaped headers, and malformed models fail loading. Literal credentials and executable secret commands are not accepted. + +This is Axl's native schema, not Pi's models.json schema. The file is user-authored, not a generated catalog or cache. Restart the daemon after editing it; `/refresh` does not reload provider configuration. The retired `custom-provider.json` path fails with migration instructions instead of silently falling back. To migrate, place its object under `providers.custom` and remove the old file after reviewing the new configuration. + +## Catalog lifecycle and updates + +Static models come from reviewed local provider-scoped source shards and overlays and are generated into the compact index and provider shards at `packages/ai/src/catalog.generated.ts` and `packages/ai/src/catalog.generated/`. Follow [`packages/ai/catalog/README.md`](../../packages/ai/catalog/README.md) for the exact update procedure. Generation is offline and deterministic. + +GitHub Copilot, OpenRouter, Cloudflare AI Gateway, and Radius have dynamic catalogs. The 35 static providers backed by models.dev also support live metadata refresh through the same reviewed normalization and policy used by generation. Ant Ling and explicit custom model lists do not have remote discovery. `axl refresh [provider-id]` and `/refresh [provider-id]` are the explicit first-party triggers. Unqualified refresh skips logged-out providers; authentication-check and refresh failures remain visible and make the CLI fail. Dynamic discovery authenticates; public models.dev requests carry no provider credentials. Refresh reads a bounded response, validates the complete candidate and provider-specific endpoint origin, writes a provider-scoped snapshot atomically, and publishes only the current generation. Dispatch revalidates endpoint policy, including restored snapshots, before attaching credentials or prompts. Cancellation, malformed responses, failed fetches, corrupt snapshots, and superseded refreshes cannot replace the last-known-good catalog. Startup may restore a validated cached snapshot without credentials or network work. Listing never refreshes. + +## Known limitations + +- The first-party product surface supports text-model sessions. OpenRouter image generation is implemented in `@axl/ai`, but no daemon, SDK, CLI, or TUI image-generation command is exposed. +- OpenAI Codex uses stateless SSE with complete prepared-history replay. It does not guess connection-scoped WebSocket continuation state. +- OpenAI Chat rejects response-side `reasoning_details` rather than silently discarding it. Request-side replay of already retained same-model signatures is supported. +- Opaque replay metadata is in-process only. Restart and persisted history reconstruction do not restore provider signatures or continuation IDs. +- Vercel OIDC is not implemented. Vercel AI Gateway API-key authentication is supported. +- Some subscription and gateway protocols do not publish complete stable wire specifications. Their behavior is pinned to the provenance recorded in the focused support documents and deterministic fixtures. +- Dynamic providers may have no selectable models before the first successful explicit refresh when no valid cached snapshot exists. +- TUI daemon cleanup is ordered before temporary-directory removal, and expanded multi-tool rendering remains bounded while retaining complete inputs and results. The test timeout and valid assertions are unchanged. + +## Deterministic verification + +Routine verification is fully local. It uses injected HTTP functions, in-memory credential and catalog stores, temporary directories, and checked-in manifests. It covers all 41 registrations, all 11 native dialects, authentication precedence and lifecycle, generated and dynamic catalogs, refresh races, offline restoration, mixed-dialect dispatch, configured Chat and Responses endpoints, redaction, and cross-provider continuation sanitization. It makes no live provider request and uses no live credential. + +Run the AI package suite with: + +```bash +node --test --test-timeout=30000 packages/ai/test/*.test.ts +``` + +Run repository gates with: + +```bash +pnpm check +reuse lint +pnpm audit --audit-level high +``` + +The exact requirement-to-test mapping is in [`deterministic-verification.md`](deterministic-verification.md). + +## Explicit opt-in live smoke process + +There is no automated live-provider suite in the repository. Live smoke testing is manual, billable, provider-dependent, and outside routine verification. It must never run in CI or as part of `pnpm check`. + +An operator who explicitly opts in should use a disposable account or least-privilege credential, choose one inexpensive model, and perform only these steps: + +1. Read the provider's current terms, pricing, data-use policy, region availability, and endpoint documentation. +2. Export only the provider variables required by the matrix, or run `axl login ` inside a trusted local terminal. +3. For a dynamic provider, run `axl refresh ` and confirm the expected model appears in `axl models `. +4. Run one minimal request: `axl print --provider --model "Reply with the word ok."` +5. Confirm text, terminal completion, usage, cost when supplied, provider identity, and absence of secrets in stderr and the session JSONL. +6. Run `axl logout `, unset exported variables, and remove any disposable provider credential according to provider policy. +7. Record provider, model, region, date, command shape, and redacted result. Never record tokens, authorization codes, prompt answers, or raw authorization headers. + +Cancellation, tools, images, long context, rate limits, and regional failover can create cost or side effects and are not part of the minimal smoke procedure. Expanding a smoke test requires a separate explicit approval and budget. Step 13 does not perform any live smoke test. diff --git a/package.json b/package.json index c4b44f2..7b799fb 100644 --- a/package.json +++ b/package.json @@ -20,7 +20,7 @@ "lint": "biome lint --error-on-warnings .", "release": "node scripts/release.ts", "release:preview": "node scripts/release.ts --preview", - "test": "pnpm build && node --test --test-timeout=30000 packages/*/test/*.test.ts packages/extensions/*/test/*.test.ts scripts/*.test.ts", + "test": "pnpm build && node --test --test-concurrency=1 --test-timeout=30000 packages/*/test/*.test.ts packages/extensions/*/test/*.test.ts scripts/*.test.ts", "typecheck": "tsc --noEmit" }, "devDependencies": { diff --git a/packages/ai/README.md b/packages/ai/README.md index a8a6584..262456d 100644 --- a/packages/ai/README.md +++ b/packages/ai/README.md @@ -1,8 +1,22 @@ + # `@axl/ai` -This package keeps provider-specific behavior outside the kernel. It defines provider and model contracts, credential lookup, thinking levels, tool dialects, deterministic test models, and the Azure OpenAI Responses adapter with Axl's built-in model catalog. +This package owns provider-specific model behavior outside the kernel. It defines provider and model contracts, credential lookup, request preparation, API dialects, deterministic test models, provider transports, and built-in provider registration. -Ordinary requests use the selected model's advertised output maximum, reduced only to fit estimated input plus a 4,096-token context reserve. The adapter sends the effective ceiling explicitly. Model HTTP transport uses Undici with a configurable five-minute default inactivity timeout for headers and response-body bytes. Streaming bytes refresh the timeout, zero disables it, and no absolute request deadline is imposed. +Supported native dialects include OpenAI Chat Completions, OpenAI Responses, Azure OpenAI Responses, OpenAI Codex Responses, Anthropic Messages, Google Generative AI, Google Vertex AI, Bedrock Converse Stream, Mistral Conversations, Gateway messages, and OpenRouter image generation. Models select their dialect through validated catalog metadata. Provider identity does not imply a dialect. + +Every dispatch passes through `prepareModelRequest()`. Preparation validates and snapshots history, verifies content-addressed images, renders tools while retaining canonical identities, fits token and reasoning budgets, validates sampling and compatibility controls, and removes foreign replay metadata. Unsupported or unsafe input fails before provider I/O. + +Provider transports enforce endpoint and header policy, bounded parsing, finite inactivity timeouts, cancellation, and bounded pre-stream retries. Credentials remain provider-owned and are never included in prompts, catalogs, diagnostics, canonical events, or public SDK projections. Construction and static listing perform no credential lookup, network request, or background work. + +Static model metadata is generated offline from reviewed local manifests and overlays. Dynamic providers and models.dev-backed built-ins use explicit, cancellable refreshes and provider-scoped last-known-good snapshots. Invalid, cancelled, corrupt, or superseded refreshes cannot replace a valid generation. + +See: + +- [`catalog/README.md`](catalog/README.md) for catalog provenance and regeneration. +- [Provider reference](../../docs/provider-support/provider-reference.md) for setup, authentication, endpoints, compatibility, and limitations. +- [Provider implementation notes](../../docs/provider-support/implementation-notes.md) for behavioral provenance and durable design decisions. +- [Deterministic verification](../../docs/provider-support/deterministic-verification.md) for the requirement-to-test map. diff --git a/packages/ai/catalog/README.md b/packages/ai/catalog/README.md new file mode 100644 index 0000000..47a95d8 --- /dev/null +++ b/packages/ai/catalog/README.md @@ -0,0 +1,39 @@ + + + +# Model catalog sources + +This directory contains reviewed inputs for Axl's generated static model catalog. Runtime catalog reads use only the generated index and provider shards under `src/catalog.generated.ts` and `src/catalog.generated/`. They do not read environment variables, credential stores, source manifests, or the network. + +## Provenance + +`sources/models-dev/manifest.json` indexes provider-scoped JSON Lines shards reduced from factual model metadata retrieved from `https://models.dev/api.json` on 2026-09-05. The upstream response SHA-256 and corresponding `anomalyco/models.dev` repository revision are recorded in the manifest. The upstream catalog is MIT licensed. Only provider identity, model identity, display name, capability flags, reasoning options, token limits, lifecycle status, and pricing fields required by Axl are retained. + +`sources/ant-ling/manifest.json` indexes the Ant Ling shard independently curated from the official API overview, OpenAI-compatible API reference, and reasoning-effort guide listed in that manifest. It contains factual compatibility metadata and no copied implementation. + +Each source shard contains exactly one canonical JSON model record per line, ordered by model ID. Each manifest orders providers by ID and records the model count and SHA-256 of every shard. Generation fails on a noncanonical line, ordering change, count mismatch, checksum mismatch, unindexed shard, or missing indexed shard. + +Pi at commit `92d8e2d17d4f357788381c49ce2cdb3f4ed1f21c` was consulted only as an architectural and behavioral reference for separating source data, provider policy, validation, and generated output. No Pi catalog data or source was copied or mechanically translated. + +## Updating + +1. Retrieve the current upstream source into a temporary location. +2. Record its retrieval time, SHA-256, and source revision. +3. Reduce it to the existing source fields for the provider IDs declared in `src/catalog-overlays.ts`, with one canonical JSON model record per line. +4. Update the provider's manifest count and shard SHA-256, while preserving deterministic provider and model ordering. +5. Review endpoint, region, dialect, reasoning, cache, and compatibility overlays against official provider documentation. +6. Run `node packages/ai/scripts/generate-catalog.ts`. +7. Review the provider-scoped source and generated diffs for unexpected endpoint, region, pricing, capability, and availability changes. +8. Run `pnpm check:generated` and the complete AI package tests. The semantic-baseline test must continue to pass unless an intentional catalog update separately reviews and updates that baseline. + +Generation is deliberately local and deterministic. It never fetches remote data and fails before writing when source records or overlays are invalid. Do not copy model data from the pinned Pi behavioral reference. + +GitHub Copilot, OpenRouter, Cloudflare AI Gateway, and Radius use dynamic provider discovery instead of this generator. Their catalogs change only through explicit `axl refresh`, are validated before publication, and are persisted as provider-scoped last-known-good snapshots. A dynamic catalog source change requires focused refresh, cancellation, malformed-response, race, persistence, and offline-restoration tests. + +## Explicit runtime refresh + +`axl refresh [provider-id]` and `/refresh [provider-id]` also refresh the 35 static providers mapped to models.dev. `src/catalog-normalization.ts` and `src/catalog-overlays.ts` are shared by generation and runtime refresh, so endpoint, dialect, cache, and compatibility policy remain reviewed local code. Remote data supplies model facts, not endpoints or credential headers. Each fetch is bounded to 32 MiB and 15 seconds and sends no provider credentials to models.dev. + +Unqualified refresh checks configured authentication and skips logged-out providers. Targeted refresh can retrieve public static metadata without a credential. Ant Ling remains a documentation-curated catalog, and user-configured providers retain their explicit model lists. Neither pretends to support remote discovery. These catalogs require a release update or a models.json edit respectively. + +Runtime refresh writes only the user's catalog cache. It does not modify checked-in source shards, generated files, or user-authored models.json. Existing registry merge semantics retain bundled models while upserting refreshed IDs; a missing upstream ID is not interpreted as an entitlement revocation. diff --git a/packages/ai/catalog/semantic-baseline.json b/packages/ai/catalog/semantic-baseline.json new file mode 100644 index 0000000..75fcacb --- /dev/null +++ b/packages/ai/catalog/semantic-baseline.json @@ -0,0 +1,8 @@ +{ + "schemaVersion": 1, + "capturedFromCommit": "0c6b036fc75ae3a7d2721a5dc7b076f385cbbb4a", + "providerCount": 41, + "staticProviderCount": 36, + "modelCount": 1102, + "stableSerializationSha256": "a5069e2017a50867f8c9846e05f56017cc037bdb4754c2aa97168746f886bc19" +} diff --git a/packages/ai/catalog/sources/ant-ling/manifest.json b/packages/ai/catalog/sources/ant-ling/manifest.json new file mode 100644 index 0000000..bc7b71a --- /dev/null +++ b/packages/ai/catalog/sources/ant-ling/manifest.json @@ -0,0 +1,24 @@ +{ + "SPDX-FileCopyrightText": "2026 Kaushik Kumar", + "SPDX-License-Identifier": "Apache-2.0", + "schemaVersion": 1, + "_provenance": { + "retrievedAt": "2026-09-05T13:49:08Z", + "sources": [ + "https://developer.ant-ling.com/en/docs/api-reference/", + "https://developer.ant-ling.com/en/docs/api-reference/openai/", + "https://developer.ant-ling.com/en/docs/tutorials/effort/" + ], + "sourceSha256": "3c715bab4b49c6b76d5f65ba7d3545ff20e9ed1fb793e9183cbf488ce47dd1ae" + }, + "providers": [ + { + "id": "ant-ling", + "name": "Ant Ling", + "documentation": "https://developer.ant-ling.com/en/docs/api-reference/", + "file": "providers/ant-ling.jsonl", + "modelCount": 4, + "sha256": "3bc11bf1537f73a210d21cebd26ad14b8ede68e0b7923d6f6b8f39c57b10c727" + } + ] +} diff --git a/packages/ai/catalog/sources/ant-ling/providers/ant-ling.jsonl b/packages/ai/catalog/sources/ant-ling/providers/ant-ling.jsonl new file mode 100644 index 0000000..6705765 --- /dev/null +++ b/packages/ai/catalog/sources/ant-ling/providers/ant-ling.jsonl @@ -0,0 +1,4 @@ +{"id":"Ling-2.6-1T","name":"Ling 2.6 1T","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":32000} +{"id":"Ling-2.6-flash","name":"Ling 2.6 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":32000} +{"id":"Ling-3.0-flash","name":"Ling 3.0 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":32000} +{"id":"Ring-2.6-1T","name":"Ring 2.6 1T","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","xhigh"]}],"contextWindow":262144,"maxOutputTokens":32000} diff --git a/packages/ai/catalog/sources/models-dev/manifest.json b/packages/ai/catalog/sources/models-dev/manifest.json new file mode 100644 index 0000000..077af24 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/manifest.json @@ -0,0 +1,286 @@ +{ + "SPDX-FileCopyrightText": "2025 models.dev contributors", + "SPDX-License-Identifier": "MIT", + "schemaVersion": 1, + "_provenance": { + "source": "https://models.dev/api.json", + "retrievedAt": "2026-09-05T13:49:08Z", + "sourceSha256": "0b09a4d8dedab6a804ca15046729bb2ec03c5f5b689b89a983ea488bb71eaeef", + "repository": "https://github.com/anomalyco/models.dev", + "repositoryCommit": "5c600a037417cf778ee6eb3ea2ce0f17abc12130" + }, + "providers": [ + { + "id": "alibaba-token-plan", + "name": "Alibaba Token Plan", + "documentation": "https://www.alibabacloud.com/help/en/model-studio/token-plan-overview", + "file": "providers/alibaba-token-plan.jsonl", + "modelCount": 19, + "sha256": "fbfbfa4988064dec2d933f5dd3bd1e1cfd39a6d84a92115633cfd6f5f574cd2f" + }, + { + "id": "alibaba-token-plan-cn", + "name": "Alibaba Token Plan (China)", + "documentation": "https://www.alibabacloud.com/help/zh/model-studio/token-plan-overview", + "file": "providers/alibaba-token-plan-cn.jsonl", + "modelCount": 19, + "sha256": "654b603f4fcf0b25c0f4f1545d379aa8dedf4ec9d4b8f49698845434dd1e7ba5" + }, + { + "id": "amazon-bedrock", + "name": "Amazon Bedrock", + "documentation": "https://docs.aws.amazon.com/bedrock/latest/userguide/models-supported.html", + "file": "providers/amazon-bedrock.jsonl", + "modelCount": 123, + "sha256": "f51e93b9cb6c0e405da2923cf7236c2962dd32260b4b809339030bdda54dc892" + }, + { + "id": "anthropic", + "name": "Anthropic", + "documentation": "https://docs.anthropic.com/en/docs/about-claude/models", + "file": "providers/anthropic.jsonl", + "modelCount": 14, + "sha256": "8a13f34f7b8cf7b7bfe7acaefc74635384343aae2e9375f3888c653e75740a2d" + }, + { + "id": "azure", + "name": "Azure", + "documentation": "https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models", + "file": "providers/azure.jsonl", + "modelCount": 83, + "sha256": "611e208dd3c117e6405a362f7d46822c047c4d54f252234b14dfc788b967d59f" + }, + { + "id": "baseten", + "name": "Baseten", + "documentation": "https://docs.baseten.co/inference/model-apis/overview", + "file": "providers/baseten.jsonl", + "modelCount": 22, + "sha256": "b201ea65669891ec98b79649bef40f301e2e65923bec40c84ca7389b9a71958a" + }, + { + "id": "cerebras", + "name": "Cerebras", + "documentation": "https://inference-docs.cerebras.ai/models/overview", + "file": "providers/cerebras.jsonl", + "modelCount": 2, + "sha256": "6331dd409c16f4b6110fbc24fb7b4627a7dcd62d8959fd49c852dbff9233e0f0" + }, + { + "id": "cloudflare-workers-ai", + "name": "Cloudflare Workers AI", + "documentation": "https://developers.cloudflare.com/workers-ai/models/", + "file": "providers/cloudflare-workers-ai.jsonl", + "modelCount": 27, + "sha256": "a07784492b5ef323984d397b6cb9d0957d4f36118ac273e9c230212e38586df1" + }, + { + "id": "deepseek", + "name": "DeepSeek", + "documentation": "https://api-docs.deepseek.com/quick_start/pricing", + "file": "providers/deepseek.jsonl", + "modelCount": 3, + "sha256": "7cb60e7ebe34bb503c99356663af7e933379814d395be5750732eccef684571a" + }, + { + "id": "fireworks-ai", + "name": "Fireworks AI", + "documentation": "https://fireworks.ai/docs/", + "file": "providers/fireworks-ai.jsonl", + "modelCount": 20, + "sha256": "8ac57532d87250a7965fa7c8c414122cb361ed2f3382c8fd4a657304cfcc0484" + }, + { + "id": "github-copilot", + "name": "GitHub Copilot", + "documentation": "https://docs.github.com/en/copilot", + "file": "providers/github-copilot.jsonl", + "modelCount": 28, + "sha256": "1c2c534cee6bbd1fe3bce25b5c1a723a2458106c85cb67726e77a667265a75e2" + }, + { + "id": "google", + "name": "Google", + "documentation": "https://ai.google.dev/gemini-api/docs/models", + "file": "providers/google.jsonl", + "modelCount": 32, + "sha256": "1f33fd212a7ce3d111ab1214b378599a890ca116a0d2f3d73c42bea146061929" + }, + { + "id": "google-vertex", + "name": "Vertex", + "documentation": "https://cloud.google.com/vertex-ai/generative-ai/docs/models", + "file": "providers/google-vertex.jsonl", + "modelCount": 43, + "sha256": "d09ebb398f824b0897b5950b8c128d6ee14e0767c7b6517d771c23e3d6e60d83" + }, + { + "id": "groq", + "name": "Groq", + "documentation": "https://console.groq.com/docs/models", + "file": "providers/groq.jsonl", + "modelCount": 12, + "sha256": "6f9ac6adf953fe3a4562750664f26bc079eb2648f57563638a3bd40179a9cda4" + }, + { + "id": "huggingface", + "name": "Hugging Face", + "documentation": "https://huggingface.co/docs/inference-providers", + "file": "providers/huggingface.jsonl", + "modelCount": 73, + "sha256": "6ea93329f6a7474443b036cf2e1f676b09284bfa620270bf7fb4a5b9945c1c59" + }, + { + "id": "kimi-for-coding", + "name": "Kimi For Coding", + "documentation": "https://www.kimi.com/code/docs/en/kimi-code/models.html", + "file": "providers/kimi-for-coding.jsonl", + "modelCount": 4, + "sha256": "f4829b1febf8d25675a93e76b5df88afdce3778f92695847bcf297727901af33" + }, + { + "id": "minimax", + "name": "MiniMax (minimax.io)", + "documentation": "https://platform.minimax.io/docs/guides/quickstart", + "file": "providers/minimax.jsonl", + "modelCount": 7, + "sha256": "dfbdd0e73d7fe4b374428958aaec80d7cf934c62b397c3133c662680a6b46c99" + }, + { + "id": "minimax-cn", + "name": "MiniMax (minimaxi.com)", + "documentation": "https://platform.minimaxi.com/docs/guides/quickstart", + "file": "providers/minimax-cn.jsonl", + "modelCount": 7, + "sha256": "dfbdd0e73d7fe4b374428958aaec80d7cf934c62b397c3133c662680a6b46c99" + }, + { + "id": "mistral", + "name": "Mistral", + "documentation": "https://docs.mistral.ai/getting-started/models/", + "file": "providers/mistral.jsonl", + "modelCount": 32, + "sha256": "15e93c1a804fe908fe33d8bca9140bf1da162addefe677e2656d8de91c3aba52" + }, + { + "id": "moonshotai", + "name": "Moonshot AI", + "documentation": "https://platform.moonshot.ai/docs/api/chat", + "file": "providers/moonshotai.jsonl", + "modelCount": 10, + "sha256": "a46467d56782c1b50ef0aa59ca38c9fbe3eeab245fa14617da125be3320cf1fb" + }, + { + "id": "moonshotai-cn", + "name": "Moonshot AI (China)", + "documentation": "https://platform.moonshot.cn/docs/api/chat", + "file": "providers/moonshotai-cn.jsonl", + "modelCount": 10, + "sha256": "a46467d56782c1b50ef0aa59ca38c9fbe3eeab245fa14617da125be3320cf1fb" + }, + { + "id": "nvidia", + "name": "Nvidia", + "documentation": "https://docs.api.nvidia.com/nim/", + "file": "providers/nvidia.jsonl", + "modelCount": 89, + "sha256": "f387d72533faa3b72e7bf223c7033ce06a2ab9259cbb916ff754c77aa42174a3" + }, + { + "id": "openai", + "name": "OpenAI", + "documentation": "https://platform.openai.com/docs/models", + "file": "providers/openai.jsonl", + "modelCount": 43, + "sha256": "dcc269308c30e644088d0cc72e29a87e5af6059d3c87b10f6824260d3e5da10f" + }, + { + "id": "opencode", + "name": "OpenCode Zen", + "documentation": "https://opencode.ai/docs/zen", + "file": "providers/opencode.jsonl", + "modelCount": 102, + "sha256": "628ec66ca5598c049cc50b0a26b74f5effe3a2366fbd9dce0d2dfb83b508976c" + }, + { + "id": "opencode-go", + "name": "OpenCode Go", + "documentation": "https://opencode.ai/docs/zen", + "file": "providers/opencode-go.jsonl", + "modelCount": 35, + "sha256": "138472ba8465f6db3b0a2acc86f503fd1d1469428b4421bf5d93c49cc240bce8" + }, + { + "id": "togetherai", + "name": "Together AI", + "documentation": "https://docs.together.ai/docs/serverless-models", + "file": "providers/togetherai.jsonl", + "modelCount": 38, + "sha256": "8ed382d1404146c8c07f52e82899e39025f6f96507b5d854e2edda32d1f25891" + }, + { + "id": "vercel", + "name": "Vercel AI Gateway", + "documentation": "https://github.com/vercel/ai/tree/5eb85cc45a259553501f535b8ac79a77d0e79223/packages/gateway", + "file": "providers/vercel.jsonl", + "modelCount": 277, + "sha256": "e0f51933d269f3404522608c37c800ca06fa21a246ff54cac45b0dfb6d4c56bf" + }, + { + "id": "xai", + "name": "xAI", + "documentation": "https://docs.x.ai/docs/models", + "file": "providers/xai.jsonl", + "modelCount": 7, + "sha256": "5e697f0657c546dbc72a1e5c71da97a2ee2d3592f1db9cc5db33d053e0655862" + }, + { + "id": "xiaomi", + "name": "Xiaomi", + "documentation": "https://platform.xiaomimimo.com/#/docs", + "file": "providers/xiaomi.jsonl", + "modelCount": 6, + "sha256": "7b86957b18eade691ead0cfca98a37bac25ceb69a155ff736406d5d163039eca" + }, + { + "id": "xiaomi-token-plan-ams", + "name": "Xiaomi Token Plan (Europe)", + "documentation": "https://platform.xiaomimimo.com/#/docs", + "file": "providers/xiaomi-token-plan-ams.jsonl", + "modelCount": 3, + "sha256": "f7d81e895628b08631a5ea3ffb0221fcc1e9db377fe50f5dba599c9eec90ebc7" + }, + { + "id": "xiaomi-token-plan-cn", + "name": "Xiaomi Token Plan (China)", + "documentation": "https://platform.xiaomimimo.com/#/docs", + "file": "providers/xiaomi-token-plan-cn.jsonl", + "modelCount": 3, + "sha256": "f7d81e895628b08631a5ea3ffb0221fcc1e9db377fe50f5dba599c9eec90ebc7" + }, + { + "id": "xiaomi-token-plan-sgp", + "name": "Xiaomi Token Plan (Singapore)", + "documentation": "https://platform.xiaomimimo.com/#/docs", + "file": "providers/xiaomi-token-plan-sgp.jsonl", + "modelCount": 3, + "sha256": "f7d81e895628b08631a5ea3ffb0221fcc1e9db377fe50f5dba599c9eec90ebc7" + }, + { + "id": "zai", + "name": "Z.AI", + "documentation": "https://docs.z.ai/guides/overview/pricing", + "file": "providers/zai.jsonl", + "modelCount": 16, + "sha256": "48d7eb4fee9f4b2127a2c08d4e1c6ecc7743503b7115f35d29100bb15e21ea65" + }, + { + "id": "zhipuai-coding-plan", + "name": "Zhipu AI Coding Plan", + "documentation": "https://docs.bigmodel.cn/cn/coding-plan/overview", + "file": "providers/zhipuai-coding-plan.jsonl", + "modelCount": 10, + "sha256": "217966fa6e83bc3275ba950978a824d69421697f45017f59c29c7144eeafde33" + } + ] +} diff --git a/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan-cn.jsonl b/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan-cn.jsonl new file mode 100644 index 0000000..9aa23b7 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan-cn.jsonl @@ -0,0 +1,19 @@ +{"id":"deepseek-v3.2","name":"DeepSeek V3.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5","name":"GLM-5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":16384,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":128000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":98304,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":196608,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.6-flash","name":"Qwen3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":131072}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.6-plus","name":"Qwen3.6 Plus","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":131072}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.7-max","name":"Qwen3.7 Max","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.7-plus","name":"Qwen3.7 Plus","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-flash","name":"Qwen3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-max","name":"Qwen3.8 Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-max-preview","name":"Qwen3.8 Max Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0},"status":"beta"} diff --git a/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan.jsonl b/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan.jsonl new file mode 100644 index 0000000..7e492d1 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/alibaba-token-plan.jsonl @@ -0,0 +1,19 @@ +{"id":"deepseek-v3.2","name":"DeepSeek V3.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5","name":"GLM-5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":16384,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":128000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens"}],"contextWindow":262144,"maxOutputTokens":98304,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":196608,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.6-flash","name":"Qwen3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":131072}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.6-plus","name":"Qwen3.6 Plus","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":131072}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.7-max","name":"Qwen3.7 Max","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.7-plus","name":"Qwen3.7 Plus","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-flash","name":"Qwen3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-max","name":"Qwen3.8 Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"qwen3.8-max-preview","name":"Qwen3.8 Max Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0},"status":"beta"} diff --git a/packages/ai/catalog/sources/models-dev/providers/amazon-bedrock.jsonl b/packages/ai/catalog/sources/models-dev/providers/amazon-bedrock.jsonl new file mode 100644 index 0000000..94c7f32 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/amazon-bedrock.jsonl @@ -0,0 +1,123 @@ +{"id":"amazon.nova-2-lite-v1:0","name":"Nova 2 Lite","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.33,"output":2.75}} +{"id":"amazon.nova-lite-v1:0","name":"Nova Lite","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"input":0.06,"output":0.24,"cache_read":0.015}} +{"id":"amazon.nova-micro-v1:0","name":"Nova Micro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.035,"output":0.14,"cache_read":0.00875}} +{"id":"amazon.nova-pro-v1:0","name":"Nova Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"input":0.8,"output":3.2,"cache_read":0.2}} +{"id":"anthropic.claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"anthropic.claude-fable-5-1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"anthropic.claude-opus-4-1-20250805-v1:0","name":"Claude Opus 4.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75},"status":"deprecated"} +{"id":"anthropic.claude-opus-4-5-20251101-v1:0","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic.claude-opus-4-6-v1","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic.claude-opus-4-7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic.claude-opus-4-8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic.claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"anthropic.claude-sonnet-4-6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"anthropic.claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"au.anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5 (AU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"au.anthropic.claude-opus-4-6-v1","name":"AU Anthropic Claude Opus 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":16.5,"output":82.5,"cache_read":1.65,"cache_write":20.625}} +{"id":"au.anthropic.claude-opus-4-8","name":"Claude Opus 4.8 (AU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"au.anthropic.claude-opus-5","name":"Claude Opus 5 (AU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"au.anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5 (AU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"au.anthropic.claude-sonnet-4-6","name":"AU Anthropic Claude Sonnet 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":3.3,"output":16.5,"cache_read":0.33,"cache_write":4.125}} +{"id":"au.anthropic.claude-sonnet-5","name":"Claude Sonnet 5 (AU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"deepseek.r1-v1:0","name":"DeepSeek-R1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":1.35,"output":5.4}} +{"id":"deepseek.v3-v1:0","name":"DeepSeek-V3.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":81920,"cost":{"input":0.58,"output":1.68}} +{"id":"deepseek.v3.2","name":"DeepSeek-V3.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":81920,"cost":{"input":0.62,"output":1.85}} +{"id":"eu.anthropic.claude-fable-5","name":"Claude Fable 5 (EU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":11,"output":55,"cache_read":1.1,"cache_write":13.75}} +{"id":"eu.anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1.1,"output":5.5,"cache_read":0.11,"cache_write":1.375}} +{"id":"eu.anthropic.claude-opus-4-5-20251101-v1:0","name":"Claude Opus 4.5 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5.5,"output":27.5,"cache_read":0.55,"cache_write":6.875}} +{"id":"eu.anthropic.claude-opus-4-6-v1","name":"Claude Opus 4.6 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5.5,"output":27.5,"cache_read":0.55,"cache_write":6.875}} +{"id":"eu.anthropic.claude-opus-4-7","name":"Claude Opus 4.7 (EU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5.5,"output":27.5,"cache_read":0.55,"cache_write":6.875}} +{"id":"eu.anthropic.claude-opus-4-8","name":"Claude Opus 4.8 (EU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5.5,"output":27.5,"cache_read":0.55,"cache_write":6.875}} +{"id":"eu.anthropic.claude-opus-5","name":"Claude Opus 5 (EU)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5.5,"output":27.5,"cache_read":0.55,"cache_write":6.875}} +{"id":"eu.anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3.3,"output":16.5,"cache_read":0.33,"cache_write":4.125}} +{"id":"eu.anthropic.claude-sonnet-4-6","name":"Claude Sonnet 4.6 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3.3,"output":16.5,"cache_read":0.33,"cache_write":4.125}} +{"id":"eu.anthropic.claude-sonnet-5","name":"Claude Sonnet 5 (EU)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2.2,"output":11,"cache_read":0.22,"cache_write":2.75}} +{"id":"global.anthropic.claude-fable-5","name":"Claude Fable 5 (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"global.anthropic.claude-fable-5-1","name":"Claude Fable 5.1 (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"global.anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"global.anthropic.claude-opus-4-5-20251101-v1:0","name":"Claude Opus 4.5 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"global.anthropic.claude-opus-4-6-v1","name":"Claude Opus 4.6 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"global.anthropic.claude-opus-4-7","name":"Claude Opus 4.7 (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"global.anthropic.claude-opus-4-8","name":"Claude Opus 4.8 (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"global.anthropic.claude-opus-5","name":"Claude Opus 5 (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"global.anthropic.claude-sonnet-4-6","name":"Claude Sonnet 4.6 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"global.anthropic.claude-sonnet-5","name":"Claude Sonnet 5 (Global)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"global.openai.gpt-5.6-luna","name":"GPT-5.6 Luna (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}}} +{"id":"global.openai.gpt-5.6-sol","name":"GPT-5.6 Sol (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.4,"cache_write":5,"tiers":[{"input":8,"output":30,"cache_read":0.8,"cache_write":10,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":8,"output":30,"cache_read":0.8,"cache_write":10}}} +{"id":"global.openai.gpt-5.6-terra","name":"GPT-5.6 Terra (Global)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":12,"cache_read":0.2,"cache_write":2.5,"tiers":[{"input":4,"output":18,"cache_read":0.4,"cache_write":5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4,"cache_write":5}}} +{"id":"google.gemma-3-12b-it","name":"Google Gemma 3 12B","toolCall":false,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":0.049999999999999996,"output":0.09999999999999999}} +{"id":"google.gemma-3-27b-it","name":"Google Gemma 3 27B Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":202752,"maxOutputTokens":8192,"cost":{"input":0.12,"output":0.2}} +{"id":"google.gemma-3-4b-it","name":"Gemma 3 4B IT","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.04,"output":0.08}} +{"id":"jp.anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5 (JP)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"jp.anthropic.claude-opus-4-7","name":"Claude Opus 4.7 (JP)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"jp.anthropic.claude-opus-4-8","name":"Claude Opus 4.8 (JP)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"jp.anthropic.claude-opus-5","name":"Claude Opus 5 (JP)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"jp.anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5 (JP)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"jp.anthropic.claude-sonnet-4-6","name":"Claude Sonnet 4.6 (JP)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"jp.anthropic.claude-sonnet-5","name":"Claude Sonnet 5 (JP)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"meta.llama3-1-70b-instruct-v1:0","name":"Llama 3.1 70B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.72,"output":0.72}} +{"id":"meta.llama3-1-8b-instruct-v1:0","name":"Llama 3.1 8B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.22,"output":0.22}} +{"id":"meta.llama3-3-70b-instruct-v1:0","name":"Llama 3.3 70B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.72,"output":0.72}} +{"id":"meta.llama4-maverick-17b-instruct-v1:0","name":"Llama 4 Maverick 17B Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"input":0.24,"output":0.97}} +{"id":"meta.llama4-scout-17b-instruct-v1:0","name":"Llama 4 Scout 17B Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":3500000,"maxOutputTokens":16384,"cost":{"input":0.17,"output":0.66}} +{"id":"minimax.minimax-m2","name":"MiniMax M2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204608,"maxOutputTokens":128000,"cost":{"input":0.3,"output":1.2}} +{"id":"minimax.minimax-m2.1","name":"MiniMax M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2}} +{"id":"minimax.minimax-m2.5","name":"MiniMax M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":196608,"maxOutputTokens":98304,"cost":{"input":0.3,"output":1.2}} +{"id":"mistral.devstral-2-123b","name":"Devstral 2 123B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"input":0.4,"output":2}} +{"id":"mistral.magistral-small-2509","name":"Magistral Small 1.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":40000,"cost":{"input":0.5,"output":1.5}} +{"id":"mistral.ministral-3-14b-instruct","name":"Ministral 14B 3.0","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.2,"output":0.2}} +{"id":"mistral.ministral-3-3b-instruct","name":"Ministral 3 3B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"input":0.1,"output":0.1}} +{"id":"mistral.ministral-3-8b-instruct","name":"Ministral 3 8B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.15,"output":0.15}} +{"id":"mistral.mistral-large-3-675b-instruct","name":"Mistral Large 3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"input":0.5,"output":1.5}} +{"id":"mistral.pixtral-large-2502-v1:0","name":"Pixtral Large (25.02)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":2,"output":6}} +{"id":"mistral.voxtral-mini-3b-2507","name":"Voxtral Mini 3B 2507","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.04,"output":0.04}} +{"id":"mistral.voxtral-small-24b-2507","name":"Voxtral Small 24B 2507","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":8192,"cost":{"input":0.15,"output":0.35}} +{"id":"moonshot.kimi-k2-thinking","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262143,"maxOutputTokens":16000,"cost":{"input":0.6,"output":2.5}} +{"id":"moonshotai.kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262143,"maxOutputTokens":16000,"cost":{"input":0.6,"output":3}} +{"id":"nvidia.nemotron-nano-12b-v2","name":"NVIDIA Nemotron Nano 12B v2 VL BF16","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.2,"output":0.6}} +{"id":"nvidia.nemotron-nano-3-30b","name":"NVIDIA Nemotron Nano 3 30B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.06,"output":0.24}} +{"id":"nvidia.nemotron-nano-9b-v2","name":"NVIDIA Nemotron Nano 9B v2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.06,"output":0.23}} +{"id":"nvidia.nemotron-super-3-120b","name":"NVIDIA Nemotron 3 Super 120B A12B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.65}} +{"id":"openai.gpt-5.4","name":"GPT-5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":272000,"maxOutputTokens":128000,"cost":{"input":2.75,"output":16.5,"cache_read":0.275}} +{"id":"openai.gpt-5.5","name":"GPT-5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":272000,"maxOutputTokens":128000,"cost":{"input":5.5,"output":33,"cache_read":0.55}} +{"id":"openai.gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.22,"output":1.32,"cache_read":0.022,"cache_write":0.275,"tiers":[{"input":0.44,"output":1.98,"cache_read":0.044,"cache_write":0.55,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.44,"output":1.98,"cache_read":0.044,"cache_write":0.55}}} +{"id":"openai.gpt-5.6-sol","name":"GPT-5.6 Sol","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4.4,"output":22,"cache_read":0.44,"cache_write":5.5,"tiers":[{"input":8.8,"output":33,"cache_read":0.88,"cache_write":11,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":8.8,"output":33,"cache_read":0.88,"cache_write":11}}} +{"id":"openai.gpt-5.6-terra","name":"GPT-5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.2,"output":13.2,"cache_read":0.22,"cache_write":2.75,"tiers":[{"input":4.4,"output":19.8,"cache_read":0.44,"cache_write":5.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4.4,"output":19.8,"cache_read":0.44,"cache_write":5.5}}} +{"id":"openai.gpt-oss-120b","name":"gpt-oss-120b","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6}} +{"id":"openai.gpt-oss-120b-1:0","name":"gpt-oss-120b","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6}} +{"id":"openai.gpt-oss-20b","name":"gpt-oss-20b","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.07,"output":0.3}} +{"id":"openai.gpt-oss-20b-1:0","name":"gpt-oss-20b","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.07,"output":0.3}} +{"id":"openai.gpt-oss-safeguard-120b","name":"GPT OSS Safeguard 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6}} +{"id":"openai.gpt-oss-safeguard-20b","name":"GPT OSS Safeguard 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.07,"output":0.2}} +{"id":"qwen.qwen3-235b-a22b-2507-v1:0","name":"Qwen3 235B A22B 2507","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.22,"output":0.88}} +{"id":"qwen.qwen3-32b-v1:0","name":"Qwen3 32B (dense)","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":16384,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6}} +{"id":"qwen.qwen3-coder-30b-a3b-v1:0","name":"Qwen3 Coder 30B A3B Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.6}} +{"id":"qwen.qwen3-coder-480b-a35b-v1:0","name":"Qwen3 Coder 480B A35B Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.22,"output":1.8}} +{"id":"qwen.qwen3-coder-next","name":"Qwen3 Coder Next","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.22,"output":1.8}} +{"id":"qwen.qwen3-next-80b-a3b","name":"Qwen/Qwen3-Next-80B-A3B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.14,"output":1.4}} +{"id":"qwen.qwen3-vl-235b-a22b","name":"Qwen/Qwen3-VL-235B-A22B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.3,"output":1.5}} +{"id":"us.anthropic.claude-fable-5","name":"Claude Fable 5 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"us.anthropic.claude-fable-5-1","name":"Claude Fable 5.1 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":11,"output":55,"cache_read":0.275,"cache_write":13.75}} +{"id":"us.anthropic.claude-haiku-4-5-20251001-v1:0","name":"Claude Haiku 4.5 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"us.anthropic.claude-opus-4-1-20250805-v1:0","name":"Claude Opus 4.1 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75},"status":"deprecated"} +{"id":"us.anthropic.claude-opus-4-5-20251101-v1:0","name":"Claude Opus 4.5 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"us.anthropic.claude-opus-4-6-v1","name":"Claude Opus 4.6 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"us.anthropic.claude-opus-4-7","name":"Claude Opus 4.7 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"us.anthropic.claude-opus-4-8","name":"Claude Opus 4.8 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"us.anthropic.claude-opus-5","name":"Claude Opus 5 (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"us.anthropic.claude-sonnet-4-5-20250929-v1:0","name":"Claude Sonnet 4.5 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"us.anthropic.claude-sonnet-4-6","name":"Claude Sonnet 4.6 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"us.anthropic.claude-sonnet-5","name":"Claude Sonnet 5 (US)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"us.deepseek.r1-v1:0","name":"DeepSeek-R1 (US)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":1.35,"output":5.4}} +{"id":"us.meta.llama4-maverick-17b-instruct-v1:0","name":"Llama 4 Maverick 17B Instruct (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"input":0.24,"output":0.97}} +{"id":"us.meta.llama4-scout-17b-instruct-v1:0","name":"Llama 4 Scout 17B Instruct (US)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":3500000,"maxOutputTokens":16384,"cost":{"input":0.17,"output":0.66}} +{"id":"writer.palmyra-x4-v1:0","name":"Palmyra X4","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":122880,"maxOutputTokens":8192,"cost":{"input":2.5,"output":10}} +{"id":"writer.palmyra-x5-v1:0","name":"Palmyra X5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1040000,"maxOutputTokens":8192,"cost":{"input":0.6,"output":6}} +{"id":"xai.grok-4.3","name":"Grok 4.3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"xai.grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2.2,"output":6.6,"cache_read":0.55}} +{"id":"zai.glm-4.7","name":"GLM-4.7","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2}} +{"id":"zai.glm-4.7-flash","name":"GLM-4.7-Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0.07,"output":0.4}} +{"id":"zai.glm-5","name":"GLM-5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":101376,"cost":{"input":1,"output":3.2}} diff --git a/packages/ai/catalog/sources/models-dev/providers/anthropic.jsonl b/packages/ai/catalog/sources/models-dev/providers/anthropic.jsonl new file mode 100644 index 0000000..45b9aa9 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/anthropic.jsonl @@ -0,0 +1,14 @@ +{"id":"claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"claude-fable-5-1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"claude-haiku-4-5","name":"Claude Haiku 4.5 (latest)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-haiku-4-5-20251001","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-opus-4-5","name":"Claude Opus 4.5 (latest)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-5-20251101","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-6","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-sonnet-4-5","name":"Claude Sonnet 4.5 (latest)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-4-5-20250929","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-4-6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} diff --git a/packages/ai/catalog/sources/models-dev/providers/azure.jsonl b/packages/ai/catalog/sources/models-dev/providers/azure.jsonl new file mode 100644 index 0000000..c748255 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/azure.jsonl @@ -0,0 +1,83 @@ +{"id":"claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5},"status":"beta"} +{"id":"claude-fable-5-1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"claude-haiku-4-5","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-mythos-5","name":"Claude Mythos 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5},"status":"beta"} +{"id":"claude-opus-4-1","name":"Claude Opus 4.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75}} +{"id":"claude-opus-4-5","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-6","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5}}} +{"id":"claude-opus-4-7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5}}} +{"id":"claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-sonnet-4-5","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-4-6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5},"status":"beta"} +{"id":"codestral-2501","name":"Codestral 25.01","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.3,"output":0.9}} +{"id":"codex-mini","name":"Codex Mini","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.5,"output":6,"cache_read":0.375},"status":"deprecated"} +{"id":"cohere-command-a","name":"Command A","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":2.5,"output":10}} +{"id":"cohere-embed-v-4-0","name":"Embed v4","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":1536,"cost":{"input":0.12,"output":0}} +{"id":"cohere-embed-v3-english","name":"Embed v3 English","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":512,"maxOutputTokens":1024,"cost":{"input":0.1,"output":0}} +{"id":"cohere-embed-v3-multilingual","name":"Embed v3 Multilingual","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":512,"maxOutputTokens":1024,"cost":{"input":0.1,"output":0}} +{"id":"deepseek-r1","name":"DeepSeek-R1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":163840,"cost":{"input":1.35,"output":5.4},"status":"deprecated"} +{"id":"deepseek-v3.2","name":"DeepSeek-V3.2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.58,"output":1.68}} +{"id":"deepseek-v3.2-speciale","name":"DeepSeek-V3.2-Speciale","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.58,"output":1.68}} +{"id":"deepseek-v4-flash","name":"DeepSeek-V4-Flash","toolCall":false,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.19,"output":0.51}} +{"id":"deepseek-v4-pro","name":"DeepSeek-V4-Pro","toolCall":false,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":1.74,"output":3.48}} +{"id":"gpt-3.5-turbo-0125","name":"GPT-3.5 Turbo 0125","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16384,"maxOutputTokens":16384,"cost":{"input":0.5,"output":1.5},"status":"deprecated"} +{"id":"gpt-3.5-turbo-1106","name":"GPT-3.5 Turbo 1106","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16384,"maxOutputTokens":16384,"cost":{"input":1,"output":2},"status":"deprecated"} +{"id":"gpt-3.5-turbo-instruct","name":"GPT-3.5 Turbo Instruct","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":4096,"maxOutputTokens":4096,"cost":{"input":1.5,"output":2},"status":"deprecated"} +{"id":"gpt-4-turbo","name":"GPT-4 Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":10,"output":30},"status":"deprecated"} +{"id":"gpt-4-turbo-vision","name":"GPT-4 Turbo Vision","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":10,"output":30},"status":"deprecated"} +{"id":"gpt-4.1","name":"GPT-4.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":2,"output":8,"cache_read":0.5},"status":"deprecated"} +{"id":"gpt-4.1-mini","name":"GPT-4.1 mini","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.4,"output":1.6,"cache_read":0.1},"status":"deprecated"} +{"id":"gpt-4.1-nano","name":"GPT-4.1 nano","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.1,"output":0.4,"cache_read":0.025},"status":"deprecated"} +{"id":"gpt-4o","name":"GPT-4o","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2.5,"output":10,"cache_read":1.25},"status":"deprecated"} +{"id":"gpt-4o-mini","name":"GPT-4o mini","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6,"cache_read":0.075},"status":"deprecated"} +{"id":"gpt-5","name":"GPT-5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.13}} +{"id":"gpt-5-codex","name":"GPT-5-Codex","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.13}} +{"id":"gpt-5-mini","name":"GPT-5 Mini","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.03}} +{"id":"gpt-5-nano","name":"GPT-5 Nano","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.05,"output":0.4,"cache_read":0.01}} +{"id":"gpt-5-pro","name":"GPT-5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high"]}],"contextWindow":400000,"maxOutputTokens":272000,"cost":{"input":15,"output":120}} +{"id":"gpt-5.1","name":"GPT-5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5.1-codex","name":"GPT-5.1 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5.1-codex-max","name":"GPT-5.1 Codex Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5.1-codex-mini","name":"GPT-5.1 Codex Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.025}} +{"id":"gpt-5.2","name":"GPT-5.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.125}} +{"id":"gpt-5.2-codex","name":"GPT-5.2 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.3-codex","name":"GPT-5.3 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.4","name":"GPT-5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25,"tiers":[{"input":5,"output":22.5,"cache_read":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":5,"output":22.5,"cache_read":0.5}}} +{"id":"gpt-5.4-mini","name":"GPT-5.4 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"gpt-5.4-nano","name":"GPT-5.4 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.25,"cache_read":0.02}} +{"id":"gpt-5.4-pro","name":"GPT-5.4 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180,"tiers":[{"input":60,"output":270,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":60,"output":270}}} +{"id":"gpt-5.5","name":"GPT-5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5,"tiers":[{"input":10,"output":45,"cache_read":1,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":10,"output":45,"cache_read":1}}} +{"id":"gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}},"status":"beta"} +{"id":"gpt-5.6-sol","name":"GPT-5.6 Sol","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":45,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":10,"output":45,"cache_read":1,"cache_write":12.5}},"status":"beta"} +{"id":"gpt-5.6-terra","name":"GPT-5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":12,"cache_read":0.2,"cache_write":2.5,"tiers":[{"input":4,"output":18,"cache_read":0.4,"cache_write":5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4,"cache_write":5}},"status":"beta"} +{"id":"gpt-chat-latest","name":"GPT Chat Latest","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":5,"output":30,"cache_read":0.5},"status":"beta"} +{"id":"grok-4-1-fast-non-reasoning","name":"Grok 4.1 Fast (Non-Reasoning)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.2,"output":0.5,"cache_read":0.05},"status":"beta"} +{"id":"grok-4-1-fast-reasoning","name":"Grok 4.1 Fast (Reasoning)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.2,"output":0.5,"cache_read":0.05},"status":"beta"} +{"id":"grok-4-20-non-reasoning","name":"Grok 4.20 (Non-Reasoning)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262000,"maxOutputTokens":8192,"cost":{"input":2,"output":6},"status":"beta"} +{"id":"grok-4-20-reasoning","name":"Grok 4.20 (Reasoning)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262000,"maxOutputTokens":8192,"cost":{"input":2,"output":6},"status":"beta"} +{"id":"grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":200000,"maxOutputTokens":128000,"status":"beta"} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":3}} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"llama-3.3-70b-instruct","name":"Llama-3.3-70B-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":0.71,"output":0.71}} +{"id":"llama-4-maverick-17b-128e-instruct-fp8","name":"Llama 4 Maverick 17B 128E Instruct FP8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"input":0.25,"output":1}} +{"id":"llama-4-scout-17b-16e-instruct","name":"Llama 4 Scout 17B 16E Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.2,"output":0.78}} +{"id":"ministral-3b","name":"Ministral 3B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.04,"output":0.04}} +{"id":"mistral-medium-2505","name":"Mistral Medium 3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.4,"output":2}} +{"id":"mistral-small-2503","name":"Mistral Small 3.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":0.1,"output":0.3}} +{"id":"model-router","name":"Model Router","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":200000,"maxOutputTokens":16384,"cost":{"input":0.14,"output":0}} +{"id":"o1","name":"o1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":15,"output":60,"cache_read":7.5},"status":"deprecated"} +{"id":"o3","name":"o3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"o3-mini","name":"o3-mini","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.55},"status":"deprecated"} +{"id":"o4-mini","name":"o4-mini","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.275},"status":"deprecated"} +{"id":"phi-4","name":"Phi-4","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.125,"output":0.5}} +{"id":"phi-4-mini","name":"Phi-4-mini","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.075,"output":0.3}} +{"id":"phi-4-mini-reasoning","name":"Phi-4-mini-reasoning","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.075,"output":0.3}} +{"id":"phi-4-multimodal","name":"Phi-4-multimodal","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0.08,"output":0.32,"input_audio":4}} +{"id":"phi-4-reasoning","name":"Phi-4-reasoning","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":32000,"maxOutputTokens":4096,"cost":{"input":0.125,"output":0.5}} +{"id":"phi-4-reasoning-plus","name":"Phi-4-reasoning-plus","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":32000,"maxOutputTokens":4096,"cost":{"input":0.125,"output":0.5}} +{"id":"text-embedding-3-large","name":"text-embedding-3-large","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8191,"maxOutputTokens":3072,"cost":{"input":0.13,"output":0}} +{"id":"text-embedding-3-small","name":"text-embedding-3-small","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8191,"maxOutputTokens":1536,"cost":{"input":0.02,"output":0}} +{"id":"text-embedding-ada-002","name":"text-embedding-ada-002","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536,"cost":{"input":0.1,"output":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/baseten.jsonl b/packages/ai/catalog/sources/models-dev/providers/baseten.jsonl new file mode 100644 index 0000000..0191934 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/baseten.jsonl @@ -0,0 +1,22 @@ +{"id":"deepseek-ai/DeepSeek-V3.1","name":"DeepSeek V3.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":164000,"maxOutputTokens":131000,"cost":{"input":0.5,"output":1.5},"status":"deprecated"} +{"id":"deepseek-ai/DeepSeek-V4-Flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"input":0.13,"output":0.26,"cache_read":0.028}} +{"id":"deepseek-ai/DeepSeek-V4-Pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":1.74,"output":3.48,"cache_read":0.145}} +{"id":"deepseek-ai/DeepSeek-V4-Pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":1.32,"output":3.96}} +{"id":"MiniMaxAI/MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204000,"maxOutputTokens":204000,"cost":{"input":0.3,"output":1.2},"status":"deprecated"} +{"id":"moonshotai/Kimi-K2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.6,"output":3,"cache_read":0.12}} +{"id":"moonshotai/Kimi-K2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"moonshotai/Kimi-K2.7-Code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"moonshotai/Kimi-K3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":3,"output":15}} +{"id":"nvidia/Nemotron-120B-A12B","name":"Nemotron Super","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":202800,"cost":{"input":0.3,"output":0.75,"cache_read":0.06}} +{"id":"nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B","name":"Nemotron Ultra","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":202800,"cost":{"input":0.6,"output":2.4,"cache_read":0.12}} +{"id":"openai/gpt-oss-120b","name":"OpenAI GPT 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":128072,"maxOutputTokens":128072,"cost":{"input":0.1,"output":0.5}} +{"id":"thinkingmachines/inkling","name":"Inkling","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":32768,"cost":{"input":1,"output":4.05}} +{"id":"thinkingmachines/inkling-small","name":"Inkling Small","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":32768,"cost":{"input":0.5,"output":1.2,"cache_read":0.1}} +{"id":"zai-org/GLM-4.7","name":"GLM 4.7","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":200000,"cost":{"input":0.6,"output":2.2,"cache_read":0.12}} +{"id":"zai-org/GLM-5","name":"GLM 5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":202800,"cost":{"input":0.95,"output":3.15,"cache_read":0.2}} +{"id":"zai-org/GLM-5.1","name":"GLM 5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":202800,"cost":{"input":1.3,"output":4.3,"cache_read":0.26}} +{"id":"zai-org/GLM-5.2","name":"GLM 5.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":1.4,"output":4.4,"cache_read":0.3}} +{"id":"zai-org/GLM-5.2-Fast","name":"GLM 5.2 Fast","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":2.1,"output":6.6,"cache_read":0.21}} +{"id":"zai-org/GLM-5.3","name":"GLM 5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":1.4,"output":4.4,"cache_read":0.14}} +{"id":"zai-org/GLM-5.3-Fast","name":"GLM 5.3 Fast","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":2.1,"output":6.6}} +{"id":"zai-org/GLM-5.3-Flash","name":"GLM 5.3 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.5}} diff --git a/packages/ai/catalog/sources/models-dev/providers/cerebras.jsonl b/packages/ai/catalog/sources/models-dev/providers/cerebras.jsonl new file mode 100644 index 0000000..faf0aca --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/cerebras.jsonl @@ -0,0 +1,2 @@ +{"id":"gemma-4-31b","name":"Gemma 4 31B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":40960,"cost":{"input":0.99,"output":1.49},"status":"beta"} +{"id":"gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":40960,"cost":{"input":0.35,"output":0.75}} diff --git a/packages/ai/catalog/sources/models-dev/providers/cloudflare-workers-ai.jsonl b/packages/ai/catalog/sources/models-dev/providers/cloudflare-workers-ai.jsonl new file mode 100644 index 0000000..bf9e070 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/cloudflare-workers-ai.jsonl @@ -0,0 +1,27 @@ +{"id":"@cf/aisingapore/gemma-sea-lion-v4-27b-it","name":"Gemma Sea Lion V4 27B It","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.351,"output":0.555}} +{"id":"@cf/deepseek-ai/deepseek-r1-distill-qwen-32b","name":"Deepseek R1 Distill Qwen 32B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":80000,"maxOutputTokens":80000,"cost":{"input":0.497,"output":4.881}} +{"id":"@cf/deepseek-ai/deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1310720,"maxOutputTokens":1048576,"cost":{"input":0.44,"output":1.32,"cache_read":0.014}} +{"id":"@cf/deepseek-ai/deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1.32,"output":3.96,"cache_read":0.044}} +{"id":"@cf/google/gemma-4-26b-a4b-it","name":"Gemma 4 26B A4B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":16384,"cost":{"input":0.1,"output":0.3}} +{"id":"@cf/ibm-granite/granite-4.0-h-micro","name":"Granite 4.0 H Micro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131000,"maxOutputTokens":131000,"cost":{"input":0.017,"output":0.112}} +{"id":"@cf/meta/llama-3.1-8b-instruct-fp8","name":"Llama 3.1 8B Instruct fp8","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"input":0.152,"output":0.287}} +{"id":"@cf/meta/llama-3.2-11b-vision-instruct","name":"Llama 3.2 11B Vision Instruct","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.0485,"output":0.676}} +{"id":"@cf/meta/llama-3.2-1b-instruct","name":"Llama 3.2 1B Instruct","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":60000,"maxOutputTokens":60000,"cost":{"input":0.027,"output":0.201}} +{"id":"@cf/meta/llama-3.2-3b-instruct","name":"Llama 3.2 3B Instruct","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":80000,"maxOutputTokens":80000,"cost":{"input":0.0509,"output":0.335}} +{"id":"@cf/meta/llama-3.3-70b-instruct-fp8-fast","name":"Llama 3.3 70B Instruct fp8 Fast","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":24000,"maxOutputTokens":24000,"cost":{"input":0.293,"output":2.253}} +{"id":"@cf/meta/llama-4-scout-17b-16e-instruct","name":"Llama 4 Scout 17B 16E Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131000,"maxOutputTokens":16384,"cost":{"input":0.27,"output":0.85}} +{"id":"@cf/meta/llama-guard-3-8b","name":"Llama Guard 3 8B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.484,"output":0.03}} +{"id":"@cf/mistralai/mistral-small-3.1-24b-instruct","name":"Mistral Small 3.1 24B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.351,"output":0.555}} +{"id":"@cf/moonshotai/kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":256000,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"@cf/moonshotai/kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"@cf/nvidia/nemotron-3-120b-a12b","name":"Nemotron 3 Super 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.5,"output":1.5}} +{"id":"@cf/openai/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.35,"output":0.75}} +{"id":"@cf/openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.2,"output":0.3}} +{"id":"@cf/qwen/qwen2.5-coder-32b-instruct","name":"Qwen2.5 Coder 32B Instruct","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.66,"output":1}} +{"id":"@cf/qwen/qwen3-30b-a3b-fp8","name":"Qwen3 30B A3b fp8","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.0509,"output":0.335}} +{"id":"@cf/qwen/qwen3.8-27b","name":"Qwen3.8 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.45,"output":3.2,"cache_read":0.05}} +{"id":"@cf/qwen/qwq-32b","name":"Qwq 32B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":24000,"maxOutputTokens":24000,"cost":{"input":0.66,"output":1}} +{"id":"@cf/zai-org/glm-4.7-flash","name":"GLM-4.7-Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.0605,"output":0.4}} +{"id":"@cf/zai-org/glm-5.2","name":"Glm 5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":256000,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"@cf/zai-org/glm-5.3","name":"Glm 5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":1310720,"maxOutputTokens":1310720,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"@cf/zai-org/glm-5.3-flash","name":"Glm 5.3 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1310720,"maxOutputTokens":1048576,"cost":{"input":0.15,"output":0.5,"cache_read":0.03}} diff --git a/packages/ai/catalog/sources/models-dev/providers/deepseek.jsonl b/packages/ai/catalog/sources/models-dev/providers/deepseek.jsonl new file mode 100644 index 0000000..f5377e5 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/deepseek.jsonl @@ -0,0 +1,3 @@ +{"id":"deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28,"reasoning":0.28,"cache_read":0.0028}} +{"id":"deepseek-v4-flash-vision-exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28,"reasoning":0.28,"cache_read":0.0028},"status":"beta"} +{"id":"deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.435,"output":0.87,"reasoning":0.87,"cache_read":0.003625}} diff --git a/packages/ai/catalog/sources/models-dev/providers/fireworks-ai.jsonl b/packages/ai/catalog/sources/models-dev/providers/fireworks-ai.jsonl new file mode 100644 index 0000000..637cb3d --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/fireworks-ai.jsonl @@ -0,0 +1,20 @@ +{"id":"accounts/fireworks/models/deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.22,"output":0.66,"cache_read":0.007}} +{"id":"accounts/fireworks/models/deepseek-v4-flash-vision-exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.22,"output":0.66,"cache_read":0.007}} +{"id":"accounts/fireworks/models/deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":1.32,"output":3.96,"cache_read":0.044}} +{"id":"accounts/fireworks/models/glm-5p2","name":"GLM 5.2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1048575,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.14}} +{"id":"accounts/fireworks/models/glm-5p3","name":"GLM 5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"accounts/fireworks/models/glm-5p3-flash","name":"GLM 5.3 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.5,"cache_read":0.03}} +{"id":"accounts/fireworks/models/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.15,"output":0.6,"cache_read":0.015}} +{"id":"accounts/fireworks/models/inkling","name":"Inkling","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1,"output":4.05,"cache_read":0.17}} +{"id":"accounts/fireworks/models/kimi-k2p6","name":"Kimi K2.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"accounts/fireworks/models/kimi-k2p7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"accounts/fireworks/models/kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"accounts/fireworks/models/minimax-m3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":512000,"maxOutputTokens":512000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"accounts/fireworks/models/muse-glimmer-30b","name":"Muse Glimmer 30B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.35,"output":1.5,"cache_read":0.04}} +{"id":"accounts/fireworks/models/nemotron-3-ultra-nvfp4","name":"Nemotron 3 Ultra 550B A55B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":128000,"cost":{"input":0.6,"output":2.4,"cache_read":0.119}} +{"id":"accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b","name":"Nemotron 3.5 Lightning 30B A3B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.05,"output":0.2,"cache_read":0.01}} +{"id":"accounts/fireworks/models/qwen3p7-plus","name":"Qwen 3.7 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.4,"output":1.6,"cache_read":0.08}} +{"id":"accounts/fireworks/models/qwen3p8-2p4t-a95b","name":"Qwen3.8 2.4T A95B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":2,"output":6,"cache_read":0.25}} +{"id":"accounts/fireworks/models/qwen3p8-max","name":"Qwen3.8 Max","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":2,"output":6,"cache_read":0.25}} +{"id":"accounts/fireworks/routers/glm-5p2-fast","name":"GLM 5.2 Fast","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1048575,"maxOutputTokens":131072,"cost":{"input":2.1,"output":6.6,"cache_read":0.21}} +{"id":"accounts/fireworks/routers/kimi-k3-fast","name":"Kimi K3 Fast","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":4.5,"output":22.5,"cache_read":0.45}} diff --git a/packages/ai/catalog/sources/models-dev/providers/github-copilot.jsonl b/packages/ai/catalog/sources/models-dev/providers/github-copilot.jsonl new file mode 100644 index 0000000..3a367b0 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/github-copilot.jsonl @@ -0,0 +1,28 @@ +{"id":"claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"claude-fable-5.1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"claude-haiku-4.5","name":"Claude Haiku 4.5 (latest)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024,"max":32000}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-opus-4.7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4.8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-sonnet-4.6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024,"max":32000}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"gemini-3.5-flash","name":"Gemini 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]},{"type":"budget_tokens","min":256,"max":24000}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1.5,"output":9,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.6-flash","name":"Gemini 3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]},{"type":"budget_tokens","min":256,"max":32000}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"gemini-3.7-flash","name":"Gemini 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"gemini-3.8-flash","name":"Gemini 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"gpt-5-mini","name":"GPT-5 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":264000,"maxOutputTokens":64000,"cost":{"input":0.25,"output":2,"cache_read":0.025}} +{"id":"gpt-5.3-codex","name":"GPT-5.3 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.4","name":"GPT-5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25,"tiers":[{"input":5,"output":22.5,"cache_read":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":5,"output":22.5,"cache_read":0.5}}} +{"id":"gpt-5.4-mini","name":"GPT-5.4 mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"gpt-5.4-nano","name":"GPT-5.4 nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.25,"cache_read":0.02}} +{"id":"gpt-5.5","name":"GPT-5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5,"tiers":[{"input":10,"output":45,"cache_read":1,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":10,"output":45,"cache_read":1}}} +{"id":"gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}}} +{"id":"gpt-5.6-sol","name":"GPT-5.6 Sol","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.4,"cache_write":5,"tiers":[{"input":8,"output":30,"cache_read":0.8,"cache_write":10,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":8,"output":30,"cache_read":0.8,"cache_write":10}}} +{"id":"gpt-5.6-terra","name":"GPT-5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":12,"cache_read":0.2,"cache_write":2.5,"tiers":[{"input":4,"output":18,"cache_read":0.4,"cache_write":5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4,"cache_write":5}}} +{"id":"gpt-6-astra","name":"GPT-6 Astra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5,"tiers":[{"input":20,"output":75,"cache_read":2,"cache_write":25,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":20,"output":75,"cache_read":2,"cache_write":25}}} +{"id":"grok-4.5","name":"Grok 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":128000,"cost":{"input":2,"output":6,"cache_read":0.5,"tiers":[{"input":4,"output":12,"cache_read":1,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":1}}} +{"id":"grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":500000,"maxOutputTokens":128000,"cost":{"input":2,"output":6,"cache_read":0.5,"tiers":[{"input":4,"output":12,"cache_read":1,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":1}}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"mai-code-1-flash-picker","name":"MAI-Code-1-Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"mai-code-1.1-flash","name":"MAI-Code-1.1-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02}} diff --git a/packages/ai/catalog/sources/models-dev/providers/google-vertex.jsonl b/packages/ai/catalog/sources/models-dev/providers/google-vertex.jsonl new file mode 100644 index 0000000..e046515 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/google-vertex.jsonl @@ -0,0 +1,43 @@ +{"id":"claude-fable-5-1@default","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"claude-fable-5@default","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"claude-haiku-4-5@20251001","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-opus-4-1@20250805","name":"Claude Opus 4.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75},"status":"deprecated"} +{"id":"claude-opus-4-5@20251101","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-6@default","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5}}} +{"id":"claude-opus-4-7@default","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5}}} +{"id":"claude-opus-4-8@default","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25,"tiers":[{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":10,"output":37.5,"cache_read":1,"cache_write":12.5}}} +{"id":"claude-opus-4@20250514","name":"Claude Opus 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75},"status":"deprecated"} +{"id":"claude-opus-5@default","name":"Claude Opus 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-sonnet-4-5@20250929","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-4-6@default","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75,"tiers":[{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5}}} +{"id":"claude-sonnet-4@20250514","name":"Claude Sonnet 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75},"status":"deprecated"} +{"id":"claude-sonnet-5@default","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"deepseek-ai/deepseek-v3.1-maas","name":"DeepSeek V3.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":32768,"cost":{"input":0.6,"output":1.7,"cache_read":0.06},"status":"deprecated"} +{"id":"deepseek-ai/deepseek-v3.2-maas","name":"DeepSeek V3.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":65536,"cost":{"input":0.56,"output":1.68,"cache_read":0.056},"status":"deprecated"} +{"id":"gemini-2.5-flash","name":"Gemini 2.5 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":0,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03,"input_audio":1}} +{"id":"gemini-2.5-flash-image","name":"Nano Banana","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.3,"output":30}} +{"id":"gemini-2.5-flash-lite","name":"Gemini 2.5 Flash-Lite","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":512,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.1,"output":0.4,"cache_read":0.01,"input_audio":0.3}} +{"id":"gemini-2.5-pro","name":"Gemini 2.5 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":128,"max":32768}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.25,"output":10,"cache_read":0.125,"tiers":[{"input":2.5,"output":15,"cache_read":0.25,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":15,"cache_read":0.25}}} +{"id":"gemini-3-flash-preview","name":"Gemini 3 Flash Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.5,"output":3,"cache_read":0.05,"input_audio":1}} +{"id":"gemini-3-pro-image","name":"Nano Banana Pro","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":65536,"maxOutputTokens":32768,"cost":{"input":2,"output":120,"cache_read":0.2}} +{"id":"gemini-3.1-flash-image","name":"Nano Banana 2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.5,"output":60,"cache_read":0.05}} +{"id":"gemini-3.1-flash-lite","name":"Gemini 3.1 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.25,"output":1.5,"cache_read":0.025,"input_audio":0.5}} +{"id":"gemini-3.1-flash-lite-preview","name":"Gemini 3.1 Flash Lite Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.25,"output":1.5,"cache_read":0.025,"input_audio":0.5},"status":"deprecated"} +{"id":"gemini-3.1-pro-preview","name":"Gemini 3.1 Pro Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-3.1-pro-preview-customtools","name":"Gemini 3.1 Pro Preview Custom Tools","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-3.5-flash","name":"Gemini 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":9,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.5-flash-lite","name":"Gemini 3.5 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"gemini-3.6-flash","name":"Gemini 3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-3.7-flash","name":"Gemini 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-3.8-flash","name":"Gemini 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-embedding-001","name":"Gemini Embedding 001","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":2048,"maxOutputTokens":1,"cost":{"input":0.15,"output":0}} +{"id":"gemini-flash-latest","name":"Gemini Flash Latest","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":9,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-flash-lite-latest","name":"Gemini Flash-Lite Latest","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.25,"output":1.5,"cache_read":0.025,"input_audio":0.5}} +{"id":"meta/llama-3.3-70b-instruct-maas","name":"Llama 3.3 70B Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.72,"output":0.72},"status":"deprecated"} +{"id":"meta/llama-4-maverick-17b-128e-instruct-maas","name":"Llama 4 Maverick 17B 128E Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":524288,"maxOutputTokens":8192,"cost":{"input":0.35,"output":1.15}} +{"id":"moonshotai/kimi-k2-thinking-maas","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.06},"status":"deprecated"} +{"id":"openai/gpt-oss-120b-maas","name":"GPT OSS 120B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.09,"output":0.36}} +{"id":"openai/gpt-oss-20b-maas","name":"GPT OSS 20B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.07,"output":0.25,"cache_read":0.007},"status":"deprecated"} +{"id":"qwen/qwen3-235b-a22b-instruct-2507-maas","name":"Qwen3 235B A22B Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0.22,"output":0.88},"status":"deprecated"} +{"id":"zai-org/glm-4.7-maas","name":"GLM-4.7","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":128000,"cost":{"input":0.6,"output":2.2,"cache_read":0.06},"status":"deprecated"} +{"id":"zai-org/glm-5-maas","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2,"cache_read":0.1},"status":"deprecated"} diff --git a/packages/ai/catalog/sources/models-dev/providers/google.jsonl b/packages/ai/catalog/sources/models-dev/providers/google.jsonl new file mode 100644 index 0000000..1403511 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/google.jsonl @@ -0,0 +1,32 @@ +{"id":"deep-research-max-preview-04-2026","name":"Deep Research Max Preview (Apr-21-2026)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"deep-research-preview-04-2026","name":"Deep Research Preview (Apr-21-2026)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-2.5-computer-use-preview-10-2025","name":"Gemini 2.5 Computer Use Preview 10-2025","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":1.25,"output":10,"tiers":[{"input":2.5,"output":15,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":15}}} +{"id":"gemini-2.5-flash","name":"Gemini 2.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":0,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03,"input_audio":1}} +{"id":"gemini-2.5-flash-image","name":"Nano Banana","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.3,"output":30,"cache_read":0.075}} +{"id":"gemini-2.5-flash-lite","name":"Gemini 2.5 Flash-Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":512,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.1,"output":0.4,"cache_read":0.01,"input_audio":0.3}} +{"id":"gemini-2.5-pro","name":"Gemini 2.5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":128,"max":32768}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.25,"output":10,"cache_read":0.125,"tiers":[{"input":2.5,"output":15,"cache_read":0.25,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":15,"cache_read":0.25}}} +{"id":"gemini-3-flash-preview","name":"Gemini 3 Flash Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.5,"output":3,"cache_read":0.05,"input_audio":1}} +{"id":"gemini-3-pro-image","name":"Nano Banana Pro","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":2,"output":120}} +{"id":"gemini-3-pro-image-preview","name":"Nano Banana Pro","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":2,"output":120}} +{"id":"gemini-3.1-flash-image","name":"Nano Banana 2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":65536,"maxOutputTokens":65536,"cost":{"input":0.5,"output":60}} +{"id":"gemini-3.1-flash-image-preview","name":"Nano Banana 2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":65536,"maxOutputTokens":65536,"cost":{"input":0.5,"output":60}} +{"id":"gemini-3.1-flash-lite","name":"Gemini 3.1 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.25,"output":1.5,"cache_read":0.025,"input_audio":0.5}} +{"id":"gemini-3.1-flash-lite-image","name":"Nano Banana 2 Lite","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":65536,"maxOutputTokens":65536,"cost":{"input":0.25,"output":30}} +{"id":"gemini-3.1-flash-lite-preview","name":"Gemini 3.1 Flash Lite Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.25,"output":1.5,"cache_read":0.025,"input_audio":0.5},"status":"deprecated"} +{"id":"gemini-3.1-flash-live-preview","name":"Gemini 3.1 Flash Live Preview","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.75,"output":4.5,"input_audio":3,"output_audio":12}} +{"id":"gemini-3.1-pro-preview","name":"Gemini 3.1 Pro Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-3.1-pro-preview-customtools","name":"Gemini 3.1 Pro Preview Custom Tools","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-3.5-flash","name":"Gemini 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":9,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.5-flash-lite","name":"Gemini 3.5 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"gemini-3.5-live-translate-preview","name":"Gemini 3.5 Live Translate Preview","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16384,"maxOutputTokens":32768,"cost":{"input":3.5,"output":21,"input_audio":3.5,"output_audio":21}} +{"id":"gemini-3.6-flash","name":"Gemini 3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-3.7-flash","name":"Gemini 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-3.8-flash","name":"Gemini 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-embedding-001","name":"Gemini Embedding 001","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":2048,"maxOutputTokens":1,"cost":{"input":0.15,"output":0}} +{"id":"gemini-embedding-2","name":"Gemini Embedding 2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1,"cost":{"input":0.2,"output":0,"input_audio":6.5}} +{"id":"gemini-flash-latest","name":"Gemini Flash Latest","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075,"input_audio":0.75}} +{"id":"gemini-flash-lite-latest","name":"Gemini Flash-Lite Latest","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"gemma-4-26b-a4b-it","name":"Gemma 4 26B A4B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":32768} +{"id":"gemma-4-31b-it","name":"Gemma 4 31B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":32768} +{"id":"lyria-3-clip-preview","name":"Lyria 3 Clip Preview","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"lyria-3-pro-preview","name":"Lyria 3 Pro Preview","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/groq.jsonl b/packages/ai/catalog/sources/models-dev/providers/groq.jsonl new file mode 100644 index 0000000..c96e5b4 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/groq.jsonl @@ -0,0 +1,12 @@ +{"id":"allam-2-7b","name":"ALLaM-2-7b","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":4096,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"groq/compound","name":"Compound","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192} +{"id":"groq/compound-mini","name":"Compound Mini","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192} +{"id":"llama-3.1-8b-instant","name":"Llama 3.1 8B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.05,"output":0.08}} +{"id":"llama-3.3-70b-versatile","name":"Llama 3.3 70B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.59,"output":0.79}} +{"id":"meta-llama/llama-prompt-guard-2-22m","name":"Llama Prompt Guard 2 22M","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":512,"maxOutputTokens":512,"cost":{"input":0.03,"output":0.03},"status":"beta"} +{"id":"meta-llama/llama-prompt-guard-2-86m","name":"Prompt Guard 2 86M","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":512,"maxOutputTokens":512,"cost":{"input":0.04,"output":0.04},"status":"beta"} +{"id":"openai/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.15,"output":0.6,"cache_read":0.075}} +{"id":"openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.075,"output":0.3,"cache_read":0.0375}} +{"id":"openai/gpt-oss-safeguard-20b","name":"Safety GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.075,"output":0.3},"status":"beta"} +{"id":"qwen/qwen3.6-27b","name":"Qwen3.6 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","default"]}],"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0.6,"output":3,"cache_read":0.3}} +{"id":"qwen/qwen3.8-27b","name":"Qwen3.8 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","default","low","medium","high"]}],"contextWindow":131042,"maxOutputTokens":16384,"cost":{"input":0.8,"output":4}} diff --git a/packages/ai/catalog/sources/models-dev/providers/huggingface.jsonl b/packages/ai/catalog/sources/models-dev/providers/huggingface.jsonl new file mode 100644 index 0000000..ce8bb55 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/huggingface.jsonl @@ -0,0 +1,73 @@ +{"id":"deepseek-ai/DeepSeek-R1","name":"DeepSeek-R1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":64000,"maxOutputTokens":32768,"cost":{"input":0.7,"output":2.5}} +{"id":"deepseek-ai/DeepSeek-R1-0528","name":"DeepSeek-R1-0528","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":163840,"cost":{"input":3,"output":5}} +{"id":"deepseek-ai/DeepSeek-V3","name":"DeepSeek-V3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":64000,"maxOutputTokens":8192,"cost":{"input":0.4,"output":1.3}} +{"id":"deepseek-ai/DeepSeek-V3-0324","name":"DeepSeek V3 0324","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":163840,"maxOutputTokens":163840,"cost":{"input":0.27,"output":1.12}} +{"id":"deepseek-ai/DeepSeek-V3.1","name":"DeepSeek-V3.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":0.27,"output":1}} +{"id":"deepseek-ai/DeepSeek-V3.2","name":"DeepSeek-V3.2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163840,"maxOutputTokens":65536,"cost":{"input":0.28,"output":0.4}} +{"id":"deepseek-ai/DeepSeek-V4-Flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28}} +{"id":"deepseek-ai/DeepSeek-V4-Flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28}} +{"id":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"input":0.44,"output":1.32}} +{"id":"deepseek-ai/DeepSeek-V4-Pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high"]}],"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"input":0.435,"output":0.87,"cache_read":0.003625}} +{"id":"deepseek-ai/DeepSeek-V4-Pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":1.32,"output":3.96}} +{"id":"google/gemma-4-26B-A4B-it","name":"Gemma 4 26B A4B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0.13,"output":0.4}} +{"id":"google/gemma-4-31B-it","name":"Gemma 4 31B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0.14,"output":0.4}} +{"id":"meta-llama/Llama-3.1-8B-Instruct","name":"Llama-3.1-8B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":4096,"cost":{"input":0.06,"output":0.06}} +{"id":"meta-llama/Llama-3.3-70B-Instruct","name":"Llama-3.3-70B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":4096,"cost":{"input":0.59,"output":0.79}} +{"id":"MiniMaxAI/MiniMax-M2","name":"MiniMax-M2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2}} +{"id":"MiniMaxAI/MiniMax-M2.1","name":"MiniMax-M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2}} +{"id":"MiniMaxAI/MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03}} +{"id":"MiniMaxAI/MiniMax-M2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"MiniMaxAI/MiniMax-M3","name":"MiniMax-M3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":524288,"maxOutputTokens":512000,"cost":{"input":0.3,"output":1.2}} +{"id":"moonshotai/Kimi-K2-Instruct","name":"Kimi-K2-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":1,"output":3}} +{"id":"moonshotai/Kimi-K2-Instruct-0905","name":"Kimi-K2-Instruct-0905","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":1,"output":3}} +{"id":"moonshotai/Kimi-K2-Thinking","name":"Kimi-K2-Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"moonshotai/Kimi-K2.5","name":"Kimi-K2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":3,"cache_read":0.1}} +{"id":"moonshotai/Kimi-K2.6","name":"Kimi-K2.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"moonshotai/Kimi-K2.7-Code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4}} +{"id":"moonshotai/Kimi-K3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":3,"output":15}} +{"id":"openai/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.25,"output":0.69}} +{"id":"openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.1,"output":0.5}} +{"id":"Qwen/Qwen2.5-Coder-32B-Instruct","name":"Qwen2.5-Coder-32B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":0.06,"output":0.2}} +{"id":"Qwen/Qwen3-235B-A22B","name":"Qwen3 235B-A22B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":40960,"maxOutputTokens":16384,"cost":{"input":0.2,"output":0.8}} +{"id":"Qwen/Qwen3-235B-A22B-Instruct-2507","name":"Qwen3 235B-A22B Instruct 2507","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0.855,"output":2.565}} +{"id":"Qwen/Qwen3-235B-A22B-Thinking-2507","name":"Qwen3-235B-A22B-Thinking-2507","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.3,"output":3}} +{"id":"Qwen/Qwen3-30B-A3B","name":"Qwen3 30B A3B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":40960,"maxOutputTokens":16384,"cost":{"input":0.12,"output":0.5}} +{"id":"Qwen/Qwen3-32B","name":"Qwen3 32B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0.29,"output":0.59}} +{"id":"Qwen/Qwen3-Coder-30B-A3B-Instruct","name":"Qwen3-Coder 30B-A3B Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.07,"output":0.26}} +{"id":"Qwen/Qwen3-Coder-480B-A35B-Instruct","name":"Qwen3-Coder-480B-A35B-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"input":2,"output":2}} +{"id":"Qwen/Qwen3-Coder-Next","name":"Qwen3-Coder-Next","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.2,"output":1.5}} +{"id":"Qwen/Qwen3-Embedding-4B","name":"Qwen 3 Embedding 4B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":2048,"cost":{"input":0.01,"output":0}} +{"id":"Qwen/Qwen3-Embedding-8B","name":"Qwen 3 Embedding 8B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":4096,"cost":{"input":0.01,"output":0}} +{"id":"Qwen/Qwen3-Next-80B-A3B-Instruct","name":"Qwen3-Next-80B-A3B-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"input":0.25,"output":1}} +{"id":"Qwen/Qwen3-Next-80B-A3B-Thinking","name":"Qwen3-Next-80B-A3B-Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.3,"output":2}} +{"id":"Qwen/Qwen3-VL-235B-A22B-Instruct","name":"Qwen3 VL 235B A22B Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.3,"output":1.5}} +{"id":"Qwen/Qwen3-VL-235B-A22B-Thinking","name":"Qwen3 VL 235B A22B Thinking","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.98,"output":3.95}} +{"id":"Qwen/Qwen3.5-122B-A10B","name":"Qwen3.5 122B-A10B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.4,"output":3.2}} +{"id":"Qwen/Qwen3.5-27B","name":"Qwen3.5 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.4}} +{"id":"Qwen/Qwen3.5-35B-A3B","name":"Qwen3.5 35B-A3B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.25,"output":2}} +{"id":"Qwen/Qwen3.5-397B-A17B","name":"Qwen3.5-397B-A17B","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0.6,"output":3.6}} +{"id":"Qwen/Qwen3.5-9B","name":"Qwen3.5 9B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.17,"output":0.25}} +{"id":"Qwen/Qwen3.6-27B","name":"Qwen3.6 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.47,"output":3.19}} +{"id":"Qwen/Qwen3.6-35B-A3B","name":"Qwen3.6 35B-A3B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.15,"output":0.95}} +{"id":"Qwen/Qwen3.8-2.4T-A95B","name":"Qwen3.8 2.4T A95B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":2.5,"output":6.25}} +{"id":"Qwen/Qwen3.8-27B","name":"Qwen3.8 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0.4,"output":3}} +{"id":"stepfun-ai/Step-3.5-Flash","name":"Step 3.5 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":256000,"cost":{"input":0.1,"output":0.3}} +{"id":"stepfun-ai/Step-3.7-Flash","name":"Step 3.7 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":256000,"cost":{"input":0.2,"output":1.15}} +{"id":"tencent/Hy3","name":"Hy3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high"]}],"contextWindow":262144,"maxOutputTokens":128000,"cost":{"input":0.14,"output":0.58}} +{"id":"thinkingmachines/Inkling","name":"Inkling","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1,"output":4.05}} +{"id":"thinkingmachines/Inkling-Small","name":"Inkling Small","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":524288,"maxOutputTokens":1048576,"cost":{"input":0.5,"output":1.2}} +{"id":"XiaomiMiMo/MiMo-V2-Flash","name":"MiMo-V2-Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":4096,"cost":{"input":0.1,"output":0.3}} +{"id":"XiaomiMiMo/MiMo-V2.5","name":"MiMo-V2.5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.4,"output":2}} +{"id":"XiaomiMiMo/MiMo-V2.5-Pro","name":"MiMo-V2.5-Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1,"output":3}} +{"id":"zai-org/GLM-4.5","name":"GLM-4.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":98304,"cost":{"input":0.6,"output":2.2}} +{"id":"zai-org/GLM-4.5-Air","name":"GLM-4.5-Air","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":98304,"cost":{"input":0.13,"output":0.85}} +{"id":"zai-org/GLM-4.5V","name":"GLM-4.5V","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":65536,"maxOutputTokens":16384,"cost":{"input":0.6,"output":1.8}} +{"id":"zai-org/GLM-4.6","name":"GLM-4.6","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.55,"output":2.2}} +{"id":"zai-org/GLM-4.6V-Flash","name":"GLM-4.6V-Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.3,"output":0.9}} +{"id":"zai-org/GLM-4.7","name":"GLM-4.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2,"cache_read":0.11}} +{"id":"zai-org/GLM-4.7-Flash","name":"GLM-4.7-Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":200000,"maxOutputTokens":128000,"cost":{"input":0,"output":0}} +{"id":"zai-org/GLM-5","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2,"cache_read":0.2}} +{"id":"zai-org/GLM-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2,"cache_read":0.2}} +{"id":"zai-org/GLM-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4}} +{"id":"zai-org/GLM-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4}} +{"id":"zai-org/GLM-5.3-Flash","name":"GLM-5.3-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.5}} diff --git a/packages/ai/catalog/sources/models-dev/providers/kimi-for-coding.jsonl b/packages/ai/catalog/sources/models-dev/providers/kimi-for-coding.jsonl new file mode 100644 index 0000000..296c497 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/kimi-for-coding.jsonl @@ -0,0 +1,4 @@ +{"id":"k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"k3-256k","name":"Kimi K3-256K","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-for-coding","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"kimi-for-coding-highspeed","name":"Kimi For Coding HighSpeed","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/minimax-cn.jsonl b/packages/ai/catalog/sources/models-dev/providers/minimax-cn.jsonl new file mode 100644 index 0000000..a254c77 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/minimax-cn.jsonl @@ -0,0 +1,7 @@ +{"id":"MiniMax-M2","name":"MiniMax-M2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2}} +{"id":"MiniMax-M2.1","name":"MiniMax-M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"MiniMax-M2.5-highspeed","name":"MiniMax-M2.5-highspeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.4,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M2.7-highspeed","name":"MiniMax-M2.7-highspeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.4,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":512000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"tiers":[{"input":0.6,"output":2.4,"cache_read":0.12,"tier":{"type":"context","size":512000}}],"context_over_200k":{"input":0.6,"output":2.4,"cache_read":0.12}}} diff --git a/packages/ai/catalog/sources/models-dev/providers/minimax.jsonl b/packages/ai/catalog/sources/models-dev/providers/minimax.jsonl new file mode 100644 index 0000000..a254c77 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/minimax.jsonl @@ -0,0 +1,7 @@ +{"id":"MiniMax-M2","name":"MiniMax-M2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2}} +{"id":"MiniMax-M2.1","name":"MiniMax-M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"MiniMax-M2.5-highspeed","name":"MiniMax-M2.5-highspeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.4,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M2.7-highspeed","name":"MiniMax-M2.7-highspeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.4,"cache_read":0.06,"cache_write":0.375}} +{"id":"MiniMax-M3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":512000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"tiers":[{"input":0.6,"output":2.4,"cache_read":0.12,"tier":{"type":"context","size":512000}}],"context_over_200k":{"input":0.6,"output":2.4,"cache_read":0.12}}} diff --git a/packages/ai/catalog/sources/models-dev/providers/mistral.jsonl b/packages/ai/catalog/sources/models-dev/providers/mistral.jsonl new file mode 100644 index 0000000..5745ddb --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/mistral.jsonl @@ -0,0 +1,32 @@ +{"id":"codestral-latest","name":"Codestral (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":4096,"cost":{"input":0.3,"output":0.9}} +{"id":"devstral-2512","name":"Devstral 2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2},"status":"deprecated"} +{"id":"devstral-latest","name":"Devstral 2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2},"status":"deprecated"} +{"id":"devstral-medium-2507","name":"Devstral Medium","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.4,"output":2},"status":"deprecated"} +{"id":"devstral-medium-latest","name":"Devstral 2 (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2},"status":"deprecated"} +{"id":"devstral-small-2505","name":"Devstral Small 2505","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.1,"output":0.3},"status":"deprecated"} +{"id":"devstral-small-2507","name":"Devstral Small","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.1,"output":0.3},"status":"deprecated"} +{"id":"labs-devstral-small-2512","name":"Devstral Small 2","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"magistral-medium-latest","name":"Magistral Medium (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2,"output":5}} +{"id":"magistral-small","name":"Magistral Small","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.5,"output":1.5}} +{"id":"ministral-3b-latest","name":"Ministral 3B (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.04,"output":0.04}} +{"id":"ministral-8b-latest","name":"Ministral 8B (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.1,"output":0.1}} +{"id":"mistral-embed","name":"Mistral Embed","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8000,"maxOutputTokens":3072,"cost":{"input":0.1,"output":0}} +{"id":"mistral-large-2411","name":"Mistral Large 2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":2,"output":6}} +{"id":"mistral-large-2512","name":"Mistral Large 3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.5,"output":1.5}} +{"id":"mistral-large-latest","name":"Mistral Large (latest)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.5,"output":1.5}} +{"id":"mistral-medium-2505","name":"Mistral Medium 3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.4,"output":2}} +{"id":"mistral-medium-2508","name":"Mistral Medium 3.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2}} +{"id":"mistral-medium-2604","name":"Mistral Medium 3.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.5,"output":7.5}} +{"id":"mistral-medium-latest","name":"Mistral Medium (latest)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.5,"output":7.5}} +{"id":"mistral-nemo","name":"Mistral Nemo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.15,"output":0.15}} +{"id":"mistral-small-2506","name":"Mistral Small 3.2","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.1,"output":0.3}} +{"id":"mistral-small-2603","name":"Mistral Small 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.15,"output":0.6}} +{"id":"mistral-small-latest","name":"Mistral Small (latest)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.15,"output":0.6}} +{"id":"open-mistral-7b","name":"Mistral 7B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8000,"maxOutputTokens":8000,"cost":{"input":0.25,"output":0.25}} +{"id":"open-mistral-nemo","name":"Open Mistral Nemo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.15,"output":0.15},"status":"deprecated"} +{"id":"open-mixtral-8x22b","name":"Mixtral 8x22B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":64000,"maxOutputTokens":64000,"cost":{"input":2,"output":6}} +{"id":"open-mixtral-8x7b","name":"Mixtral 8x7B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"input":0.7,"output":0.7}} +{"id":"pixtral-12b","name":"Pixtral 12B","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.15,"output":0.15}} +{"id":"pixtral-large-latest","name":"Pixtral Large (latest)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":2,"output":6}} +{"id":"voxtral-small-latest","name":"Voxtral Small (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"input":0.1,"output":0.3}} +{"id":"zai-glm-5-2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.14},"status":"beta"} diff --git a/packages/ai/catalog/sources/models-dev/providers/moonshotai-cn.jsonl b/packages/ai/catalog/sources/models-dev/providers/moonshotai-cn.jsonl new file mode 100644 index 0000000..89171c5 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/moonshotai-cn.jsonl @@ -0,0 +1,10 @@ +{"id":"kimi-k2-0711-preview","name":"Kimi K2 0711","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-0905-preview","name":"Kimi K2 0905","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-thinking","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-thinking-turbo","name":"Kimi K2 Thinking Turbo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.15,"output":8,"cache_read":0.15}} +{"id":"kimi-k2-turbo-preview","name":"Kimi K2 Turbo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":2.4,"output":10,"cache_read":0.6}} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":3,"cache_read":0.1}} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"kimi-k2.7-code-highspeed","name":"Kimi K2.7 Code HighSpeed","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.9,"output":8,"cache_read":0.38}} +{"id":"kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} diff --git a/packages/ai/catalog/sources/models-dev/providers/moonshotai.jsonl b/packages/ai/catalog/sources/models-dev/providers/moonshotai.jsonl new file mode 100644 index 0000000..89171c5 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/moonshotai.jsonl @@ -0,0 +1,10 @@ +{"id":"kimi-k2-0711-preview","name":"Kimi K2 0711","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-0905-preview","name":"Kimi K2 0905","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-thinking","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":2.5,"cache_read":0.15}} +{"id":"kimi-k2-thinking-turbo","name":"Kimi K2 Thinking Turbo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.15,"output":8,"cache_read":0.15}} +{"id":"kimi-k2-turbo-preview","name":"Kimi K2 Turbo","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":2.4,"output":10,"cache_read":0.6}} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.6,"output":3,"cache_read":0.1}} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"kimi-k2.7-code-highspeed","name":"Kimi K2.7 Code HighSpeed","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":1.9,"output":8,"cache_read":0.38}} +{"id":"kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} diff --git a/packages/ai/catalog/sources/models-dev/providers/nvidia.jsonl b/packages/ai/catalog/sources/models-dev/providers/nvidia.jsonl new file mode 100644 index 0000000..aa08b84 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/nvidia.jsonl @@ -0,0 +1,89 @@ +{"id":"abacusai/dracarys-llama-3.1-70b-instruct","name":"dracarys-llama-3.1-70b-instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"baai/bge-m3","name":"BGE M3","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1024,"cost":{"input":0,"output":0}} +{"id":"bytedance/seed-oss-36b-instruct","name":"ByteDance-Seed/Seed-OSS-36B-Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0,"output":0}} +{"id":"deepseek-ai/deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028}} +{"id":"deepseek-ai/deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0}} +{"id":"deepseek-ai/deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"input":0.435,"output":0.87,"cache_read":0.003625}} +{"id":"deepseek-ai/deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0,"output":0}} +{"id":"google/gemma-2-2b-it","name":"Gemma 2 2b It","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"google/gemma-3-12b-it","name":"Gemma 3 12B IT","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"google/gemma-3-4b-it","name":"Gemma 3 4B IT","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"google/gemma-3n-e2b-it","name":"Gemma 3n E2b It","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"google/gemma-3n-e4b-it","name":"Gemma 3n E4b It","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"google/gemma-4-31b-it","name":"Gemma-4-31B-IT","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"google/google-paligemma","name":"paligemma","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"meta/esm2-650m","name":"esm2-650m","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"meta/esmfold","name":"esmfold","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.1-70b-instruct","name":"Llama 3.1 70b Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.1-8b-instruct","name":"Llama 3.1 8B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.2-11b-vision-instruct","name":"Llama 3.2 11b Vision Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.2-1b-instruct","name":"Llama 3.2 1b Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.2-3b-instruct","name":"Llama 3.2 3B Instruct","toolCall":false,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32000,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.2-90b-vision-instruct","name":"Llama-3.2-90B-Vision-Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"meta/llama-3.3-70b-instruct","name":"Llama 3.3 70b Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-4-maverick-17b-128e-instruct","name":"Llama 4 Maverick 17b 128e Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-guard-4-12b","name":"Llama Guard 4 12B","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"meta/muse-glimmer-30b","name":"Muse Glimmer 30B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","max"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} +{"id":"microsoft/phi-4-mini-instruct","name":"Phi-4-Mini","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"microsoft/phi-4-multimodal-instruct","name":"Phi 4 Multimodal","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"minimaxai/minimax-m2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} +{"id":"minimaxai/minimax-m3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"mistralai/magistral-small-2506","name":"Magistral Small 2506","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0,"output":0}} +{"id":"mistralai/ministral-14b-instruct-2512","name":"Ministral 3 14B Instruct 2512","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-7b-instruct-v0.3","name":"Mistral-7B-Instruct-v0.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":65536,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-large-3-675b-instruct-2512","name":"Mistral Large 3 675B Instruct 2512","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-medium-3-instruct","name":"Mistral Medium 3","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-medium-3.5-128b","name":"Mistral Medium 3.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-nemotron","name":"mistral-nemotron","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"mistralai/mistral-small-4-119b-2603","name":"mistral-small-4-119b-2603","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"mistralai/mixtral-8x22b-instruct","name":"Mistral: Mixtral 8x22B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":65536,"maxOutputTokens":13108,"cost":{"input":0,"output":0}} +{"id":"mistralai/mixtral-8x7b-instruct","name":"Mistral: Mixtral 8x7B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"moonshotai/kimi-k2-instruct-0905","name":"Kimi K2 0905","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"moonshotai/kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"moonshotai/kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} +{"id":"nvidia/bevformer","name":"bevformer","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/cosmos-reason2-8b","name":"Cosmos Reason2 8B","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"nvidia/gliner-pii","name":"gliner-pii","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3_2-nemoretriever-300m-embed-v1","name":"llama-3_2-nemoretriever-300m-embed-v1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":2048,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.1-nemotron-70b-instruct","name":"Llama 3.1 Nemotron 70B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.1-nemotron-nano-8b-v1","name":"Llama 3.1 Nemotron Nano 8B v1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.1-nemotron-nano-vl-8b-v1","name":"Llama 3.1 Nemotron Nano VL 8B v1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":32768,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.1-nemotron-safety-guard-8b-v3","name":"llama-3.1-nemotron-safety-guard-8b-v3","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.1-nemotron-ultra-253b-v1","name":"Llama 3.1 Nemotron Ultra 253B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.3-nemotron-super-49b-v1","name":"Llama 3.3 Nemotron Super 49B v1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-3.3-nemotron-super-49b-v1.5","name":"Llama 3.3 Nemotron Super 49B v1.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-nemotron-embed-vl-1b-v2","name":"llama-nemotron-embed-vl-1b-v2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":32768,"maxOutputTokens":2048,"cost":{"input":0,"output":0}} +{"id":"nvidia/llama-nemotron-rerank-vl-1b-v2","name":"llama-nemotron-rerank-vl-1b-v2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-3-content-safety","name":"nemotron-3-content-safety","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-3-nano-30b-a3b","name":"nemotron-3-nano-30b-a3b","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning","name":"Nemotron 3 Nano Omni","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":-1,"max":32768}],"contextWindow":256000,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-3-super-120b-a12b","name":"Nemotron 3 Super","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.2,"output":0.8}} +{"id":"nvidia/nemotron-3-ultra-550b-a55b","name":"Nemotron 3 Ultra 550B A55B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0.5,"output":2.5,"cache_read":0.15}} +{"id":"nvidia/nemotron-3.5-lightning-30b-a3b","name":"Nemotron 3.5 Lightning 30B A3B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-content-safety-reasoning-4b","name":"nemotron-content-safety-reasoning-4b","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-mini-4b-instruct","name":"nemotron-mini-4b-instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-nano-12b-v2-vl","name":"Nemotron Nano 12B v2 VL","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0,"output":0}} +{"id":"nvidia/nemotron-voicechat","name":"nemotron-voicechat","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/nv-embed-v1","name":"nv-embed-v1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":2048,"cost":{"input":0,"output":0}} +{"id":"nvidia/nv-embedcode-7b-v1","name":"nv-embedcode-7b-v1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":2048,"cost":{"input":0,"output":0}} +{"id":"nvidia/nvidia-nemotron-nano-9b-v2","name":"nvidia-nemotron-nano-9b-v2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} +{"id":"nvidia/rerank-qa-mistral-4b","name":"rerank-qa-mistral-4b","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/riva-translate-4b-instruct-v1.1","name":"riva-translate-4b-instruct-v1_1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"nvidia/sparsedrive","name":"sparsedrive","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/streampetr","name":"streampetr","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/studiovoice","name":"studiovoice","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"nvidia/usdcode","name":"usdcode","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"openai/gpt-oss-120b","name":"GPT-OSS-120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0,"output":0}} +{"id":"poolside/laguna-xs-2.1","name":"Laguna XS 2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"qwen/qwen2.5-coder-32b-instruct","name":"Qwen2.5 Coder 32b Instruct","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"qwen/qwen3-coder-480b-a35b-instruct","name":"Qwen3 Coder 480B A35B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"input":0,"output":0}} +{"id":"qwen/qwen3-next-80b-a3b-instruct","name":"Qwen3-Next-80B-A3B-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"qwen/qwen3.5-122b-a10b","name":"Qwen3.5 122B-A10B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0,"output":0}} +{"id":"qwen/qwen3.5-397b-a17b","name":"Qwen3.5-397B-A17B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"sarvamai/sarvam-m","name":"sarvam-m","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"stepfun-ai/step-3.5-flash","name":"Step 3.5 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"stepfun-ai/step-3.7-flash","name":"Step 3.7 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh","max"]}],"contextWindow":256000,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"thinkingmachines/inkling","name":"Inkling","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":16384,"cost":{"input":0,"output":0}} +{"id":"upstage/solar-10.7b-instruct","name":"solar-10.7b-instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0,"output":0}} +{"id":"z-ai/glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/openai.jsonl b/packages/ai/catalog/sources/models-dev/providers/openai.jsonl new file mode 100644 index 0000000..47605d5 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/openai.jsonl @@ -0,0 +1,43 @@ +{"id":"gpt-3.5-turbo","name":"GPT-3.5-turbo","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16385,"maxOutputTokens":4096,"cost":{"input":0.5,"output":1.5,"cache_read":0},"status":"deprecated"} +{"id":"gpt-4","name":"GPT-4","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":8192,"cost":{"input":30,"output":60},"status":"deprecated"} +{"id":"gpt-4-turbo","name":"GPT-4 Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":10,"output":30},"status":"deprecated"} +{"id":"gpt-4.1","name":"GPT-4.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"gpt-4.1-mini","name":"GPT-4.1 mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.4,"output":1.6,"cache_read":0.1}} +{"id":"gpt-4.1-nano","name":"GPT-4.1 nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.1,"output":0.4,"cache_read":0.025},"status":"deprecated"} +{"id":"gpt-4o","name":"GPT-4o","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2.5,"output":10,"cache_read":1.25}} +{"id":"gpt-4o-2024-05-13","name":"GPT-4o (2024-05-13)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":5,"output":15},"status":"deprecated"} +{"id":"gpt-4o-2024-08-06","name":"GPT-4o (2024-08-06)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2.5,"output":10,"cache_read":1.25}} +{"id":"gpt-4o-2024-11-20","name":"GPT-4o (2024-11-20)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2.5,"output":10,"cache_read":1.25}} +{"id":"gpt-4o-mini","name":"GPT-4o mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6,"cache_read":0.075}} +{"id":"gpt-5","name":"GPT-5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5-mini","name":"GPT-5 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.025}} +{"id":"gpt-5-nano","name":"GPT-5 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.05,"output":0.4,"cache_read":0.005}} +{"id":"gpt-5-pro","name":"GPT-5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high"]}],"contextWindow":400000,"maxOutputTokens":272000,"cost":{"input":15,"output":120}} +{"id":"gpt-5.1","name":"GPT-5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5.2","name":"GPT-5.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.2-chat-latest","name":"GPT-5.2 Chat","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium"]}],"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":1.75,"output":14,"cache_read":0.175},"status":"deprecated"} +{"id":"gpt-5.2-pro","name":"GPT-5.2 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":21,"output":168}} +{"id":"gpt-5.3-chat-latest","name":"GPT-5.3 Chat (latest)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":1.75,"output":14,"cache_read":0.175},"status":"deprecated"} +{"id":"gpt-5.3-codex","name":"GPT-5.3 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.3-codex-spark","name":"GPT-5.3 Codex Spark","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":128000,"maxOutputTokens":32000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.4","name":"GPT-5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25,"tiers":[{"input":5,"output":22.5,"cache_read":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":5,"output":22.5,"cache_read":0.5}}} +{"id":"gpt-5.4-mini","name":"GPT-5.4 mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"gpt-5.4-nano","name":"GPT-5.4 nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.25,"cache_read":0.02}} +{"id":"gpt-5.4-pro","name":"GPT-5.4 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180,"tiers":[{"input":60,"output":270,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":60,"output":270}}} +{"id":"gpt-5.5","name":"GPT-5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5,"tiers":[{"input":10,"output":45,"cache_read":1,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":10,"output":45,"cache_read":1}}} +{"id":"gpt-5.5-pro","name":"GPT-5.5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180,"tiers":[{"input":60,"output":270,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":60,"output":270}}} +{"id":"gpt-5.6","name":"GPT-5.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.4,"cache_write":5,"tiers":[{"input":8,"output":30,"cache_read":0.8,"cache_write":10,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":8,"output":30,"cache_read":0.8,"cache_write":10}}} +{"id":"gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}}} +{"id":"gpt-5.6-sol","name":"GPT-5.6 Sol","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.4,"cache_write":5,"tiers":[{"input":8,"output":30,"cache_read":0.8,"cache_write":10,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":8,"output":30,"cache_read":0.8,"cache_write":10}}} +{"id":"gpt-5.6-terra","name":"GPT-5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":12,"cache_read":0.2,"cache_write":2.5,"tiers":[{"input":4,"output":18,"cache_read":0.4,"cache_write":5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4,"cache_write":5}}} +{"id":"gpt-6-astra","name":"GPT-6 Astra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5,"tiers":[{"input":20,"output":75,"cache_read":2,"cache_write":25,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":20,"output":75,"cache_read":2,"cache_write":25}}} +{"id":"gpt-realtime-2.1","name":"GPT-Realtime-2.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":128000,"maxOutputTokens":32000,"cost":{"input":4,"output":24,"cache_read":0.4,"input_audio":32,"output_audio":64}} +{"id":"o1","name":"o1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":15,"output":60,"cache_read":7.5},"status":"deprecated"} +{"id":"o1-pro","name":"o1-pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":150,"output":600},"status":"deprecated"} +{"id":"o3","name":"o3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"o3-mini","name":"o3-mini","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.55},"status":"deprecated"} +{"id":"o3-pro","name":"o3-pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":20,"output":80}} +{"id":"o4-mini","name":"o4-mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.275},"status":"deprecated"} +{"id":"text-embedding-3-large","name":"text-embedding-3-large","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8191,"maxOutputTokens":3072,"cost":{"input":0.13,"output":0}} +{"id":"text-embedding-3-small","name":"text-embedding-3-small","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8191,"maxOutputTokens":1536,"cost":{"input":0.02,"output":0}} +{"id":"text-embedding-ada-002","name":"text-embedding-ada-002","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536,"cost":{"input":0.1,"output":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/opencode-go.jsonl b/packages/ai/catalog/sources/models-dev/providers/opencode-go.jsonl new file mode 100644 index 0000000..a59e4cc --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/opencode-go.jsonl @@ -0,0 +1,35 @@ +{"id":"deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.22,"output":0.66,"cache_read":0.007}} +{"id":"deepseek-v4-flash-vision-exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.22,"output":0.66,"cache_read":0.007}} +{"id":"deepseek-v4-pro","name":"DeepSeek V4 Pro (New)","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.66,"output":1.98,"cache_read":0.022}} +{"id":"glm-5","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":32768,"cost":{"input":1,"output":3.2,"cache_read":0.2},"status":"deprecated"} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":32768,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.3-flash","name":"GLM-5.3-Flash (2x usage)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.075,"output":0.25,"cache_read":0.015}} +{"id":"gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}}} +{"id":"grok-4.5","name":"Grok 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.3,"tiers":[{"input":4,"output":12,"cache_read":0.6,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":0.6}},"status":"deprecated"} +{"id":"grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.5,"tiers":[{"input":4,"output":12,"cache_read":1,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":1}}} +{"id":"hy3","name":"Hy3","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high"]}],"contextWindow":256000,"maxOutputTokens":128000,"cost":{"input":0.14,"output":0.58,"cache_read":0.035}} +{"id":"hy4-preview","name":"Hy4 preview","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":1024000,"maxOutputTokens":64000,"cost":{"input":0.834,"output":2.501,"cache_read":0.042}} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.6,"output":3,"cache_read":0.1},"status":"deprecated"} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"longcat-2.0","name":"LongCat-2.0","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.006}} +{"id":"mimo-v2-omni","name":"MiMo V2 Omni","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":128000,"cost":{"input":0.4,"output":2,"cache_read":0.08},"status":"deprecated"} +{"id":"mimo-v2-pro","name":"MiMo V2 Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":128000,"cost":{"input":1,"output":3,"cache_read":0.2,"tiers":[{"input":2,"output":6,"cache_read":0.4,"tier":{"type":"context","size":256000}}],"context_over_200k":{"input":2,"output":6,"cache_read":0.4}},"status":"deprecated"} +{"id":"mimo-v2.5","name":"MiMo V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028}} +{"id":"mimo-v2.5-pro","name":"MiMo V2.5 Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":128000,"cost":{"input":0.435,"output":0.87,"cache_read":0.003625}} +{"id":"minimax-m2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":65536,"cost":{"input":0.3,"output":1.2,"cache_read":0.03},"status":"deprecated"} +{"id":"minimax-m2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"minimax-m3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"tiers":[{"input":0.6,"output":2.4,"cache_read":0.12,"tier":{"type":"context","size":512000}}],"context_over_200k":{"input":0.6,"output":2.4,"cache_read":0.12}}} +{"id":"muse-spark-1.2-contributor","name":"Muse Spark 1.2 Contributor","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.1,"output":0.2,"cache_read":0.002}} +{"id":"muse-spark-1.3-contributor","name":"Muse Spark 1.3 Contributor","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.1,"output":0.2,"cache_read":0.002}} +{"id":"omen-alpha","name":"Omen Alpha","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high"]}],"contextWindow":500000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":0.66,"cache_read":0.04}} +{"id":"ox-alpha-free","name":"Ox Alpha Free (Unlimited)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"qwen3.5-plus","name":"Qwen3.5 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":81920}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25},"status":"deprecated"} +{"id":"qwen3.6-plus","name":"Qwen3.6 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":81920}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0.5,"output":3,"cache_read":0.05,"cache_write":0.625,"tiers":[{"input":2,"output":6,"cache_read":0.2,"cache_write":2.5,"tier":{"type":"context","size":256000}}],"context_over_200k":{"input":2,"output":6,"cache_read":0.2,"cache_write":2.5}}} +{"id":"qwen3.7-max","name":"Qwen3.7 Max","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":2.5,"output":7.5,"cache_read":0.5,"cache_write":3.125}} +{"id":"qwen3.7-plus","name":"Qwen3.7 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0.4,"output":1.6,"cache_read":0.04,"cache_write":0.5,"tiers":[{"input":1.2,"output":4.8,"cache_read":0.12,"cache_write":1.5,"tier":{"type":"context","size":256000}}],"context_over_200k":{"input":1.2,"output":4.8,"cache_read":0.12,"cache_write":1.5}}} +{"id":"qwen3.8-flash","name":"Qwen3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.47,"cache_read":0.016,"cache_write":0.2}} +{"id":"qwen3.8-max","name":"Qwen3.8 Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","max":262144}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":2,"output":6,"cache_read":0.25,"cache_write":2.5}} diff --git a/packages/ai/catalog/sources/models-dev/providers/opencode.jsonl b/packages/ai/catalog/sources/models-dev/providers/opencode.jsonl new file mode 100644 index 0000000..f4590c3 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/opencode.jsonl @@ -0,0 +1,102 @@ +{"id":"big-pickle","name":"Big Pickle","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"claude-3-5-haiku","name":"Claude Haiku 3.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":200000,"maxOutputTokens":8192,"cost":{"input":0.8,"output":4,"cache_read":0.08,"cache_write":1},"status":"deprecated"} +{"id":"claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"claude-fable-5-1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"claude-haiku-4-5","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"claude-opus-4-1","name":"Claude Opus 4.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75},"status":"deprecated"} +{"id":"claude-opus-4-5","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-6","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-4-8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"claude-sonnet-4","name":"Claude Sonnet 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75,"tiers":[{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5}}} +{"id":"claude-sonnet-4-5","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75,"tiers":[{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5}}} +{"id":"claude-sonnet-4-6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","max"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28,"cache_read":0.028}} +{"id":"deepseek-v4-flash-free","name":"DeepSeek V4 Flash Free","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":200000,"maxOutputTokens":128000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"deepseek-v4-flash-vision-exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28,"cache_read":0.028}} +{"id":"deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":1.74,"output":3.84,"cache_read":0.145}} +{"id":"gemini-3-flash","name":"Gemini 3 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.5,"output":3,"cache_read":0.05}} +{"id":"gemini-3-pro","name":"Gemini 3 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}},"status":"deprecated"} +{"id":"gemini-3.1-pro","name":"Gemini 3.1 Pro Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":2,"output":12,"cache_read":0.2,"tiers":[{"input":4,"output":18,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":18,"cache_read":0.4}}} +{"id":"gemini-3.5-flash","name":"Gemini 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":9,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.5-flash-lite","name":"Gemini 3.5 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"gemini-3.6-flash","name":"Gemini 3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":7.5,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.7-flash","name":"Gemini 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":7.5,"cache_read":0.15,"input_audio":1.5}} +{"id":"gemini-3.8-flash","name":"Gemini 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.5,"output":7.5,"cache_read":0.15,"input_audio":1.5}} +{"id":"glm-4.6","name":"GLM-4.6","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2,"cache_read":0.1},"status":"deprecated"} +{"id":"glm-4.7","name":"GLM-4.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2,"cache_read":0.1},"status":"deprecated"} +{"id":"glm-4.7-free","name":"GLM-4.7 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"glm-5","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2,"cache_read":0.2}} +{"id":"glm-5-free","name":"GLM-5 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"glm-5.3-flash","name":"GLM-5.3-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.5,"cache_read":0.03}} +{"id":"gpt-5","name":"GPT-5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.07,"output":8.5,"cache_read":0.107}} +{"id":"gpt-5-codex","name":"GPT-5 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.07,"output":8.5,"cache_read":0.107}} +{"id":"gpt-5-nano","name":"GPT-5 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.05,"output":0.4,"cache_read":0.005}} +{"id":"gpt-5.1","name":"GPT-5.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.07,"output":8.5,"cache_read":0.107}} +{"id":"gpt-5.1-codex","name":"GPT-5.1 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.07,"output":8.5,"cache_read":0.107}} +{"id":"gpt-5.1-codex-max","name":"GPT-5.1 Codex Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"gpt-5.1-codex-mini","name":"GPT-5.1 Codex Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.025}} +{"id":"gpt-5.2","name":"GPT-5.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.2-codex","name":"GPT-5.2 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.3-codex","name":"GPT-5.3 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.3-codex-spark","name":"GPT-5.3 Codex Spark","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"gpt-5.4","name":"GPT-5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25,"tiers":[{"input":5,"output":22.5,"cache_read":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":5,"output":22.5,"cache_read":0.5}}} +{"id":"gpt-5.4-mini","name":"GPT-5.4 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"gpt-5.4-nano","name":"GPT-5.4 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.25,"cache_read":0.02}} +{"id":"gpt-5.4-pro","name":"GPT-5.4 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180,"cache_read":30}} +{"id":"gpt-5.5","name":"GPT-5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5,"tiers":[{"input":10,"output":45,"cache_read":1,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":10,"output":45,"cache_read":1}}} +{"id":"gpt-5.5-pro","name":"GPT-5.5 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180,"cache_read":30}} +{"id":"gpt-5.6-luna","name":"GPT-5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25,"tiers":[{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":0.4,"output":1.8,"cache_read":0.04,"cache_write":0.5}}} +{"id":"gpt-5.6-sol","name":"GPT-5.6 Sol (50% Off)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5,"tiers":[{"input":4,"output":15,"cache_read":0.4,"cache_write":5,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":4,"output":15,"cache_read":0.4,"cache_write":5}}} +{"id":"gpt-5.6-terra","name":"GPT-5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25,"cache_write":3.125,"tiers":[{"input":5,"output":22.5,"cache_read":0.5,"cache_write":6.25,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":5,"output":22.5,"cache_read":0.5,"cache_write":6.25}}} +{"id":"gpt-6-astra","name":"GPT-6 Astra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5,"tiers":[{"input":20,"output":75,"cache_read":2,"cache_write":25,"tier":{"type":"context","size":272000}}],"context_over_200k":{"input":20,"output":75,"cache_read":2,"cache_write":25}}} +{"id":"grok-4.5","name":"Grok 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.3,"tiers":[{"input":4,"output":12,"cache_read":0.6,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":0.6}}} +{"id":"grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.5,"tiers":[{"input":4,"output":12,"cache_read":1,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":1}}} +{"id":"grok-build-0.1","name":"Grok Build 0.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":1,"output":2,"cache_read":0.2}} +{"id":"grok-code","name":"Grok Code Fast 1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0},"status":"deprecated"} +{"id":"hy3-free","name":"Hy3 Free","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":190000,"maxOutputTokens":64000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"hy3-preview-free","name":"Hy3 preview Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":64000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"kimi-k2","name":"Kimi K2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2.5,"cache_read":0.4},"status":"deprecated"} +{"id":"kimi-k2-thinking","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.4,"output":2.5,"cache_read":0.4},"status":"deprecated"} +{"id":"kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.6,"output":3,"cache_read":0.08}} +{"id":"kimi-k2.5-free","name":"Kimi K2.5 Free","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"laguna-s-2.1-free","name":"Laguna S 2.1 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"ling-2.6-flash-free","name":"Ling 2.6 Flash Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262100,"maxOutputTokens":32800,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"ling-3.0-flash-fin-free","name":"Ling 3.0 Flash Fin Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"ling-3.0-flash-free","name":"Ling-3.0-flash Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"ling-3.0-tiny-free","name":"Ling-3.0-tiny Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"longcat-2.0-free","name":"LongCat-2.0 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2-flash-free","name":"MiMo V2 Flash Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2-omni-free","name":"MiMo V2 Omni Free","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":64000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2-pro-free","name":"MiMo V2 Pro Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":64000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2.5-free","name":"MiMo V2.5 Free","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"minimax-m2.1","name":"MiniMax-M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.1},"status":"deprecated"} +{"id":"minimax-m2.1-free","name":"MiniMax-M2.1 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"minimax-m2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"minimax-m2.5-free","name":"MiniMax-M2.5 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"minimax-m2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"minimax-m3","name":"MiniMax-M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":512000,"maxOutputTokens":128000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"minimax-m3-free","name":"MiniMax-M3 Free","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":32000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"muse-spark-1.2","name":"Muse Spark 1.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1.25,"output":4.25,"cache_read":0.15}} +{"id":"muse-spark-1.2-contributor-free","name":"Muse Spark 1.2 Free","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"muse-spark-1.3","name":"Muse Spark 1.3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1.25,"output":4.25,"cache_read":0.15}} +{"id":"muse-spark-1.3-contributor-free","name":"Muse Spark 1.3 Free","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"nemotron-3-super-free","name":"Nemotron 3 Super Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":128000,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"nemotron-3-ultra-free","name":"Nemotron 3 Ultra Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"nemotron-3.5-lightning-free","name":"Nemotron 3.5 Lightning Free","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"north-mini-code-free","name":"North Mini Code Free","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":256000,"maxOutputTokens":64000,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"qwen3-coder","name":"Qwen3 Coder","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.45,"output":1.8},"status":"deprecated"} +{"id":"qwen3.5-plus","name":"Qwen3.5 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":81920}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25}} +{"id":"qwen3.6-plus","name":"Qwen3.6 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":81920}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.5,"output":3,"cache_read":0.05,"cache_write":0.625}} +{"id":"qwen3.6-plus-free","name":"Qwen3.6 Plus Free","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","max":81920}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"ring-2.6-1t-free","name":"Ring 2.6 1T Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262000,"maxOutputTokens":66000,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"trinity-large-preview-free","name":"Trinity Large Preview","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0,"output":0},"status":"deprecated"} +{"id":"x-preview-f-free","name":"Ox Alpha Free (Unlimited)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} diff --git a/packages/ai/catalog/sources/models-dev/providers/togetherai.jsonl b/packages/ai/catalog/sources/models-dev/providers/togetherai.jsonl new file mode 100644 index 0000000..eb38d38 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/togetherai.jsonl @@ -0,0 +1,38 @@ +{"id":"deepcogito/cogito-v2-1-671b","name":"Cogito v2.1 671B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":163840,"maxOutputTokens":163840,"cost":{"input":1.25,"output":1.25}} +{"id":"deepseek-ai/DeepSeek-R1","name":"DeepSeek-R1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":163839,"maxOutputTokens":163839,"cost":{"input":3,"output":7},"status":"deprecated"} +{"id":"deepseek-ai/DeepSeek-V3","name":"DeepSeek-V3","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":1.25,"output":1.25},"status":"deprecated"} +{"id":"deepseek-ai/DeepSeek-V3-1","name":"DeepSeek V3.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.6,"output":1.7},"status":"deprecated"} +{"id":"deepseek-ai/DeepSeek-V4-Flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.14,"output":0.28,"cache_read":0.03}} +{"id":"deepseek-ai/DeepSeek-V4-Pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":512000,"maxOutputTokens":384000,"cost":{"input":1.74,"output":3.48,"cache_read":0.2}} +{"id":"deepseek-ai/DeepSeek-V4-Pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"input":1.32,"output":3.96,"cache_read":0.13}} +{"id":"essentialai/Rnj-1-Instruct","name":"Rnj-1 Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.15,"output":0.15},"status":"deprecated"} +{"id":"google/gemma-3n-E4B-it","name":"Gemma 3N E4B Instruct","toolCall":false,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.06,"output":0.12}} +{"id":"google/gemma-4-31B-it","name":"Gemma 4 31B Instruct","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.39,"output":0.97}} +{"id":"LiquidAI/LFM2-24B-A2B","name":"LFM2-24B-A2B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.03,"output":0.12}} +{"id":"meta-llama/Llama-3.3-70B-Instruct-Turbo","name":"Llama 3.3 70B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":1.04,"output":1.04}} +{"id":"meta-llama/Meta-Llama-3-8B-Instruct-Lite","name":"Meta Llama 3 8B Instruct Lite","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":8192,"cost":{"input":0.14,"output":0.14}} +{"id":"MiniMaxAI/MiniMax-M2.5","name":"MiniMax-M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06},"status":"deprecated"} +{"id":"MiniMaxAI/MiniMax-M2.7","name":"MiniMax-M2.7","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"MiniMaxAI/MiniMax-M3","name":"MiniMax-M3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":524288,"maxOutputTokens":250000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"moonshotai/Kimi-K2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.5,"output":2.8},"status":"deprecated"} +{"id":"moonshotai/Kimi-K2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":131000,"cost":{"input":1.2,"output":4.5,"cache_read":0.2}} +{"id":"moonshotai/Kimi-K2.7-Code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.95,"output":4,"cache_read":0.19}} +{"id":"moonshotai/Kimi-K3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"nvidia/nemotron-3-ultra-550b-a55b","name":"Nemotron 3 Ultra 550B A55B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":512300,"maxOutputTokens":512300,"cost":{"input":0.6,"output":3.6,"cache_read":0.2}} +{"id":"openai/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.6}} +{"id":"openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.05,"output":0.2}} +{"id":"pearl-ai/gemma-4-31b-it","name":"Pearl AI Gemma 4 31B Instruct","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":32000,"maxOutputTokens":32000,"cost":{"input":0.28,"output":0.86}} +{"id":"Qwen/Qwen2.5-7B-Instruct-Turbo","name":"Qwen 2.5 7B Instruct Turbo","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"input":0.3,"output":0.3}} +{"id":"Qwen/Qwen3-235B-A22B-Instruct-2507-tput","name":"Qwen3 235B A22B Instruct 2507 FP8","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.2,"output":0.6},"status":"deprecated"} +{"id":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","name":"Qwen3 Coder 480B A35B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":2,"output":2},"status":"deprecated"} +{"id":"Qwen/Qwen3-Coder-Next-FP8","name":"Qwen3 Coder Next FP8","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.5,"output":1.2},"status":"deprecated"} +{"id":"Qwen/Qwen3.5-397B-A17B","name":"Qwen3.5 397B A17B","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":130000,"cost":{"input":0.6,"output":3.6,"cache_read":0.35},"status":"deprecated"} +{"id":"Qwen/Qwen3.5-9B","name":"Qwen3.5 9B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.17,"output":0.25}} +{"id":"Qwen/Qwen3.6-Plus","name":"Qwen3.6 Plus","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":500000,"cost":{"input":0.5,"output":3}} +{"id":"Qwen/Qwen3.7-Max","name":"Qwen3.7 Max","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":500000,"cost":{"input":1.25,"output":3.75,"cache_read":0.125}} +{"id":"thinkingmachines/Inkling","name":"Inkling","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["max","xhigh","high","medium","low","none"]}],"contextWindow":524288,"maxOutputTokens":131072,"cost":{"input":1,"output":4.05,"cache_read":0.17}} +{"id":"zai-org/GLM-5","name":"GLM-5","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2},"status":"deprecated"} +{"id":"zai-org/GLM-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202752,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26},"status":"deprecated"} +{"id":"zai-org/GLM-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","max"]}],"contextWindow":512000,"maxOutputTokens":164000,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"zai-org/GLM-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"zai-org/GLM-5.3-Flash","name":"GLM-5.3-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048575,"maxOutputTokens":400000,"cost":{"input":0.15,"output":0.5,"cache_read":0.03}} diff --git a/packages/ai/catalog/sources/models-dev/providers/vercel.jsonl b/packages/ai/catalog/sources/models-dev/providers/vercel.jsonl new file mode 100644 index 0000000..9ea1aae --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/vercel.jsonl @@ -0,0 +1,277 @@ +{"id":"alibaba/qwen-3-14b","name":"Qwen3-14B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":40960,"maxOutputTokens":16384,"cost":{"input":0.12,"output":0.24}} +{"id":"alibaba/qwen-3-235b","name":"Qwen3 235B A22B Instruct 2507","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":16384,"cost":{"input":0.22,"output":0.88}} +{"id":"alibaba/qwen-3-30b","name":"Qwen3-30B-A3B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":40960,"maxOutputTokens":16384,"cost":{"input":0.12,"output":0.5}} +{"id":"alibaba/qwen-3-32b","name":"Qwen 3.32B","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":38912}],"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.16,"output":0.64}} +{"id":"alibaba/qwen-3.6-max-preview","name":"Qwen 3.6 Max Preview","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":131072}],"contextWindow":240000,"maxOutputTokens":64000,"cost":{"input":1.3,"output":7.8,"cache_read":0.26,"cache_write":1.625}} +{"id":"alibaba/qwen3-235b-a22b-thinking","name":"Qwen3 235B A22B Thinking 2507","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1,"max":81920}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.4,"output":4}} +{"id":"alibaba/qwen3-coder","name":"Qwen3 Coder 480B A35B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":1.5,"output":7.5,"cache_read":0.3}} +{"id":"alibaba/qwen3-coder-30b-a3b","name":"Qwen 3 Coder 30B A3B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":8192,"cost":{"input":0.15,"output":0.6}} +{"id":"alibaba/qwen3-coder-next","name":"Qwen3 Coder Next","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.5,"output":1.2}} +{"id":"alibaba/qwen3-coder-plus","name":"Qwen3 Coder Plus","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":1,"output":5,"cache_read":0.2}} +{"id":"alibaba/qwen3-embedding-0.6b","name":"Qwen3 Embedding 0.6B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768} +{"id":"alibaba/qwen3-embedding-4b","name":"Qwen3 Embedding 4B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768} +{"id":"alibaba/qwen3-embedding-8b","name":"Qwen3 Embedding 8B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768} +{"id":"alibaba/qwen3-max","name":"Qwen3 Max","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":1.2,"output":6,"cache_read":0.24}} +{"id":"alibaba/qwen3-max-preview","name":"Qwen3 Max Preview","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":1.2,"output":6,"cache_read":0.24}} +{"id":"alibaba/qwen3-max-thinking","name":"Qwen 3 Max Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1,"max":81920}],"contextWindow":256000,"maxOutputTokens":65536,"cost":{"input":1.2,"output":6,"cache_read":0.24}} +{"id":"alibaba/qwen3-next-80b-a3b-instruct","name":"Qwen3 Next 80B A3B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.15,"output":1.2}} +{"id":"alibaba/qwen3-next-80b-a3b-thinking","name":"Qwen3 Next 80B A3B Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.15,"output":1.2}} +{"id":"alibaba/qwen3-vl-235b-a22b-instruct","name":"Qwen3 VL 235B A22B Instruct","toolCall":false,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":129024,"cost":{"input":0.4,"output":1.6}} +{"id":"alibaba/qwen3-vl-instruct","name":"Qwen3 VL Instruct","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":131072,"maxOutputTokens":129024,"cost":{"input":0.4,"output":1.6}} +{"id":"alibaba/qwen3-vl-thinking","name":"Qwen3 VL Thinking","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1,"max":81920}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.4,"output":4}} +{"id":"alibaba/qwen3.5-flash","name":"Qwen 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":81920}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.1,"output":0.4,"cache_read":0.001,"cache_write":0.125}} +{"id":"alibaba/qwen3.5-plus","name":"Qwen 3.5 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":81920}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.4,"output":2.4,"cache_read":0.04,"cache_write":0.5}} +{"id":"alibaba/qwen3.6-27b","name":"Qwen 3.6 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":131072}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.6,"output":3.6}} +{"id":"alibaba/qwen3.6-plus","name":"Qwen 3.6 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":131072}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.5,"output":3,"cache_read":0.1,"cache_write":0.625}} +{"id":"alibaba/qwen3.7-flash","name":"Qwen 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":991000,"maxOutputTokens":64000,"cost":{"input":0.03,"output":0.13,"cache_read":0.006,"cache_write":0.038}} +{"id":"alibaba/qwen3.7-max","name":"Qwen 3.7 Max","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":262144}],"contextWindow":991000,"maxOutputTokens":64000,"cost":{"input":2.5,"output":7.5,"cache_read":0.5,"cache_write":3.125}} +{"id":"alibaba/qwen3.7-plus","name":"Qwen 3.7 Plus","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":262144}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.4,"output":1.6,"cache_read":0.08,"cache_write":0.5}} +{"id":"alibaba/qwen3.8-2.4t-a95b","name":"Qwen3.8 2.4T A95B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":262144,"maxOutputTokens":128000,"cost":{"input":2,"output":6,"cache_read":0.25}} +{"id":"alibaba/qwen3.8-27b","name":"Qwen3.8 27B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.5,"output":3,"cache_read":0.1,"cache_write":0.625}} +{"id":"alibaba/qwen3.8-flash","name":"Qwen 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens"}],"contextWindow":991000,"maxOutputTokens":128000,"cost":{"input":0.16,"output":0.47,"cache_read":0.016,"cache_write":0.2}} +{"id":"alibaba/qwen3.8-flash-next","name":"Qwen 3.8 Flash Next","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0.12,"output":0.4,"cache_read":0.01}} +{"id":"alibaba/qwen3.8-max","name":"Qwen 3.8 Max","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":6,"cache_read":0.25,"cache_write":2.5}} +{"id":"alibaba/qwen3.8-max-0902","name":"Qwen3.8 Max 0902","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","xhigh"]},{"type":"budget_tokens","min":0,"max":262144}],"contextWindow":991000,"maxOutputTokens":128000,"cost":{"input":2,"output":6,"cache_read":0.25,"cache_write":2.5}} +{"id":"amazon/nova-2-lite","name":"Nova 2 Lite","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":0.3,"output":2.5,"cache_read":0.075}} +{"id":"amazon/nova-lite","name":"Nova Lite","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"input":0.06,"output":0.24,"cache_read":0.015}} +{"id":"amazon/nova-micro","name":"Nova Micro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.035,"output":0.14,"cache_read":0.00875}} +{"id":"amazon/nova-pro","name":"Nova Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"input":0.8,"output":3.2,"cache_read":0.2}} +{"id":"amazon/titan-embed-text-v2","name":"Titan Text Embeddings V2","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"anthropic/claude-3-haiku","name":"Claude Haiku 3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":200000,"maxOutputTokens":4096,"cost":{"input":0.25,"output":1.25,"cache_read":0.03,"cache_write":0.3}} +{"id":"anthropic/claude-fable-5","name":"Claude Fable 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"anthropic/claude-fable-5.1","name":"Claude Fable 5.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":0.25,"cache_write":12.5}} +{"id":"anthropic/claude-haiku-4.5","name":"Claude Haiku 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":1,"output":5,"cache_read":0.1,"cache_write":1.25}} +{"id":"anthropic/claude-opus-4","name":"Claude Opus 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":200000,"maxOutputTokens":8192,"cost":{"input":15,"output":75,"cache_read":1.5,"cache_write":18.75}} +{"id":"anthropic/claude-opus-4.5","name":"Claude Opus 4.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1024}],"contextWindow":200000,"maxOutputTokens":64000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic/claude-opus-4.6","name":"Claude Opus 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic/claude-opus-4.7","name":"Claude Opus 4.7","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic/claude-opus-4.8","name":"Claude Opus 4.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic/claude-opus-4.8-fast","name":"Claude Opus 4.8 (Fast)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"anthropic/claude-opus-5","name":"Claude Opus 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":25,"cache_read":0.5,"cache_write":6.25}} +{"id":"anthropic/claude-opus-5-fast","name":"Claude Opus 5 (Fast)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5}} +{"id":"anthropic/claude-sonnet-4","name":"Claude Sonnet 4","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":8192,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"anthropic/claude-sonnet-4.5","name":"Claude Sonnet 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75}} +{"id":"anthropic/claude-sonnet-4.6","name":"Claude Sonnet 4.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high"]},{"type":"budget_tokens","min":1024}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":3,"output":15,"cache_read":0.3,"cache_write":3.75,"tiers":[{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":6,"output":22.5,"cache_read":0.6,"cache_write":7.5}}} +{"id":"anthropic/claude-sonnet-5","name":"Claude Sonnet 5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"arcee-ai/trinity-large-thinking","name":"Trinity Large Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":262100,"maxOutputTokens":80000,"cost":{"input":0.25,"output":0.8999999999999999}} +{"id":"bytedance/seed-1.6","name":"Seed 1.6","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0.25,"output":2,"cache_read":0.05}} +{"id":"bytedance/seed-1.8","name":"Seed 1.8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":64000,"cost":{"input":0.25,"output":2,"cache_read":0.05}} +{"id":"cohere/command-a","name":"Command A","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":8000,"cost":{"input":2.5,"output":10}} +{"id":"cohere/embed-v4.0","name":"Embed v4.0","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":1536} +{"id":"cohere/rerank-v3.5","name":"Cohere Rerank 3.5","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":4096,"maxOutputTokens":4096} +{"id":"cohere/rerank-v4-fast","name":"Cohere Rerank 4 Fast","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000} +{"id":"cohere/rerank-v4-pro","name":"Cohere Rerank 4 Pro","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000} +{"id":"deepseek/deepseek-r1","name":"DeepSeek-R1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":1.35,"output":5.4}} +{"id":"deepseek/deepseek-v3.1","name":"DeepSeek-V3.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":163840,"maxOutputTokens":128000,"cost":{"input":0.25,"output":0.95,"cache_read":0.13}} +{"id":"deepseek/deepseek-v3.1-terminus","name":"DeepSeek V3.1 Terminus","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":65536,"cost":{"input":0.27,"output":1,"cache_read":0.135}} +{"id":"deepseek/deepseek-v3.2","name":"DeepSeek V3.2","toolCall":false,"structuredOutput":true,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8000,"cost":{"input":0.28,"output":0.42,"cache_read":0.028}} +{"id":"deepseek/deepseek-v3.2-thinking","name":"DeepSeek V3.2 Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":128000,"maxOutputTokens":8000,"cost":{"input":0.62,"output":1.85}} +{"id":"deepseek/deepseek-v4-flash","name":"DeepSeek V4 Flash","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.13,"output":0.26,"cache_read":0.028}} +{"id":"deepseek/deepseek-v4-flash-0731","name":"DeepSeek V4 Flash 0731","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.076,"output":0.153,"cache_read":0.014}} +{"id":"deepseek/deepseek-v4-flash-vision-exp","name":"DeepSeek V4 Flash Vision Exp","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0.22,"output":0.66,"cache_read":0.007}} +{"id":"deepseek/deepseek-v4-pro","name":"DeepSeek V4 Pro","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.66,"output":1.98,"cache_read":0.022}} +{"id":"deepseek/deepseek-v4-pro-0813","name":"DeepSeek V4 Pro 0813","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"input":0.66,"output":1.98,"cache_read":0.066}} +{"id":"google/gemini-2.5-flash","name":"Gemini 2.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":0,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03,"input_audio":1}} +{"id":"google/gemini-2.5-flash-image","name":"Nano Banana (Gemini 2.5 Flash Image)","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":32768,"maxOutputTokens":65536,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"google/gemini-2.5-flash-lite","name":"Gemini 2.5 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":512,"max":24576}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":0.1,"output":0.4,"cache_read":0.01}} +{"id":"google/gemini-2.5-pro","name":"Gemini 2.5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"budget_tokens","min":128,"max":32768}],"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"input":1.25,"output":10,"cache_read":0.125,"tiers":[{"input":2.5,"output":15,"cache_read":0.25,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":15,"cache_read":0.25}}} +{"id":"google/gemini-3-flash","name":"Gemini 3 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"input":0.5,"output":3,"cache_read":0.05}} +{"id":"google/gemini-3-pro-image","name":"Nano Banana Pro","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":65536,"maxOutputTokens":32768,"cost":{"input":2,"output":12,"cache_read":0.2}} +{"id":"google/gemini-3.1-flash-image","name":"Nano Banana 2","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.5,"output":3,"cache_read":0.05}} +{"id":"google/gemini-3.1-flash-image-preview","name":"Gemini 3.1 Flash Image Preview (Nano Banana 2)","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","high"]}],"contextWindow":131072,"maxOutputTokens":32768,"cost":{"input":0.5,"output":3,"cache_read":0.05}} +{"id":"google/gemini-3.1-flash-lite","name":"Gemini 3.1 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"input":0.25,"output":1.5,"cache_read":0.03}} +{"id":"google/gemini-3.1-flash-lite-image","name":"Gemini 3.1 Flash Lite Image (Nano Banana 2 Lite)","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":65536,"maxOutputTokens":4096,"cost":{"input":0.25,"output":1.5,"cache_read":0.03}} +{"id":"google/gemini-3.1-pro-preview","name":"Gemini 3.1 Pro Preview","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":2,"output":12,"cache_read":0.2}} +{"id":"google/gemini-3.5-flash","name":"Gemini 3.5 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":1.5,"output":9,"cache_read":0.15}} +{"id":"google/gemini-3.5-flash-lite","name":"Gemini 3.5 Flash Lite","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"input":0.3,"output":2.5,"cache_read":0.03}} +{"id":"google/gemini-3.6-flash","name":"Gemini 3.6 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"google/gemini-3.7-flash","name":"Gemini 3.7 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"google/gemini-3.8-flash","name":"Gemini 3.8 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"input":0.75,"output":3.75,"cache_read":0.075}} +{"id":"google/gemini-embedding-001","name":"Gemini Embedding 001","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"google/gemini-omni-flash-preview","name":"Gemini Omni Flash Preview","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":57920,"cost":{"input":1.5,"output":9}} +{"id":"google/gemma-4-26b-a4b-it","name":"Gemma 4 26B A4B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.15,"output":0.6,"cache_read":0.015}} +{"id":"google/gemma-4-31b-it","name":"Gemma 4 31B IT","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.14,"output":0.4}} +{"id":"google/text-embedding-005","name":"Text Embedding 005","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"google/text-multilingual-embedding-002","name":"Text Multilingual Embedding 002","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"inception/mercury-2","name":"Mercury 2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":0.75,"cache_read":0.024999999999999998}} +{"id":"inception/mercury-coder-small","name":"Mercury Coder Small Beta","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":16384,"cost":{"input":0.25,"output":1}} +{"id":"inclusionai/ling-3.0-flash","name":"Ling 3.0 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0.06,"output":0.18,"cache_read":0.012}} +{"id":"inclusionai/ling-3.0-flash-fin","name":"Ling 3.0 Flash Fin","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0,"output":0}} +{"id":"inclusionai/ling-3.0-flash-fin-free","name":"Ling 3.0 Flash Fin (Free)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0,"output":0}} +{"id":"inclusionai/ling-3.0-flash-sante","name":"Ling 3.0 Flash Sante","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0,"output":0}} +{"id":"inclusionai/ling-3.0-flash-sante-free","name":"Ling 3.0 Flash Sante (Free)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0,"output":0}} +{"id":"interfaze/interfaze-beta","name":"Interfaze Beta","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":32000,"cost":{"input":1.5,"output":3.5}} +{"id":"kwaipilot/kat-coder-air-v2.5","name":"Kat Coder Air V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":80000,"cost":{"input":0.15,"output":0.6,"cache_read":0.03}} +{"id":"kwaipilot/kat-coder-pro-v1","name":"KAT-Coder-Pro V1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"kwaipilot/kat-coder-pro-v2","name":"Kat Coder Pro V2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"kwaipilot/kat-coder-pro-v2.5","name":"Kat Coder Pro V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":80000,"cost":{"input":0.74,"output":2.96,"cache_read":0.15}} +{"id":"meta/llama-3.1-70b","name":"Llama 3.1 70B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.72,"output":0.72}} +{"id":"meta/llama-3.1-8b","name":"Llama 3.1 8B Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"input":0.22,"output":0.22}} +{"id":"meta/llama-3.3-70b","name":"Llama-3.3-70B-Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-4-maverick","name":"Llama-4-Maverick-17B-128E-Instruct-FP8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/llama-4-scout","name":"Llama-4-Scout-17B-16E-Instruct-FP8","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":0,"output":0}} +{"id":"meta/muse-glimmer-30b","name":"Muse Glimmer 30B","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.35,"output":1.5,"cache_read":0.04}} +{"id":"meta/muse-spark-1.1","name":"Muse Spark 1.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1.25,"output":4.25,"cache_read":0.15}} +{"id":"meta/muse-spark-1.2","name":"Muse Spark 1.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1.25,"output":4.25,"cache_read":0.15}} +{"id":"meta/muse-spark-1.2-contributor","name":"Muse Spark 1.2 Contributor","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0.1,"output":0.2,"cache_read":0.002}} +{"id":"meta/muse-spark-1.3","name":"Muse Spark 1.3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh","max"]}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":1.25,"output":4.25,"cache_read":0.15}} +{"id":"meta/muse-spark-1.3-contributor","name":"Muse Spark 1.3 Contributor","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0.1,"output":0.2,"cache_read":0.002}} +{"id":"minimax/minimax-m2","name":"MiniMax M2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":205000,"maxOutputTokens":205000,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"minimax/minimax-m2.1","name":"MiniMax M2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"minimax/minimax-m2.1-lightning","name":"MiniMax M2.1 Lightning","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.3,"output":2.4,"cache_read":0.03,"cache_write":0.375}} +{"id":"minimax/minimax-m2.5","name":"MiniMax M2.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131000,"cost":{"input":0.3,"output":1.2,"cache_read":0.03,"cache_write":0.375}} +{"id":"minimax/minimax-m2.5-highspeed","name":"MiniMax M2.5 High Speed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131000,"cost":{"input":0.6,"output":2.4,"cache_read":0.03,"cache_write":0.375}} +{"id":"minimax/minimax-m2.7","name":"Minimax M2.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06,"cache_write":0.375}} +{"id":"minimax/minimax-m2.7-free","name":"Minimax M2.7 (Free)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":196608,"maxOutputTokens":196608,"cost":{"input":0,"output":0}} +{"id":"minimax/minimax-m2.7-highspeed","name":"MiniMax M2.7 High Speed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":204800,"maxOutputTokens":131100,"cost":{"input":0.6,"output":2.4,"cache_read":0.06,"cache_write":0.375}} +{"id":"minimax/minimax-m3","name":"MiniMax M3","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":512000,"maxOutputTokens":512000,"cost":{"input":0.3,"output":1.2,"cache_read":0.06}} +{"id":"minimax/minimax-m3-free","name":"MiniMax M3 (Free)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"mistral/codestral","name":"Codestral (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":4096,"cost":{"input":0.3,"output":0.9}} +{"id":"mistral/codestral-embed","name":"Codestral Embed","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"mistral/devstral-2","name":"Devstral 2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.4,"output":2}} +{"id":"mistral/devstral-small-2","name":"Devstral Small 2","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.1,"output":0.3}} +{"id":"mistral/ministral-14b","name":"Ministral 14B","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.2,"output":0.2}} +{"id":"mistral/ministral-3b","name":"Ministral 3B (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.04,"output":0.04}} +{"id":"mistral/ministral-8b","name":"Ministral 8B (latest)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.1,"output":0.1}} +{"id":"mistral/mistral-embed","name":"Mistral Embed","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"mistral/mistral-large-3","name":"Mistral Large 3","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.5,"output":1.5}} +{"id":"mistral/mistral-medium","name":"Mistral Medium 3.1","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":64000,"cost":{"input":0.4,"output":2}} +{"id":"mistral/mistral-medium-3.5","name":"Mistral Medium Latest","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":1.5,"output":7.5}} +{"id":"mistral/mistral-nemo","name":"Mistral Nemo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.15,"output":0.15}} +{"id":"mistral/mistral-small","name":"Mistral Small (latest)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":32000,"maxOutputTokens":4000,"cost":{"input":0.1,"output":0.3}} +{"id":"mistral/pixtral-12b","name":"Pixtral 12B","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"input":0.15,"output":0.15}} +{"id":"moonshotai/kimi-k2","name":"Kimi K2 Instruct","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.57,"output":2.3}} +{"id":"moonshotai/kimi-k2-thinking","name":"Kimi K2 Thinking","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":216144,"maxOutputTokens":216144,"cost":{"input":0.47,"output":2,"cache_read":0.141}} +{"id":"moonshotai/kimi-k2.5","name":"Kimi K2.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262114,"maxOutputTokens":262114,"cost":{"input":0.6,"output":3,"cache_read":0.1}} +{"id":"moonshotai/kimi-k2.6","name":"Kimi K2.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262000,"maxOutputTokens":262000,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"moonshotai/kimi-k2.7-code","name":"Kimi K2.7 Code","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":32768,"cost":{"input":0.95,"output":4,"cache_read":0.16}} +{"id":"moonshotai/kimi-k2.7-code-highspeed","name":"Kimi K2.7 Code High Speed","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":262144,"maxOutputTokens":32768,"cost":{"input":1.9,"output":8,"cache_read":0.38}} +{"id":"moonshotai/kimi-k3","name":"Kimi K3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":3,"output":15,"cache_read":0.3}} +{"id":"moonshotai/kimi-k3-fast","name":"Kimi K3 Fast","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":4.5,"output":22.5,"cache_read":0.45}} +{"id":"morph/morph-v3-fast","name":"Morph v3 Fast","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16000,"maxOutputTokens":16000,"cost":{"input":0.8,"output":1.2}} +{"id":"morph/morph-v3-large","name":"Morph v3 Large","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"input":0.9,"output":1.9}} +{"id":"nvidia/nemotron-3-nano-30b-a3b","name":"Nemotron 3 Nano 30B A3B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.05,"output":0.24}} +{"id":"nvidia/nemotron-3-super-120b-a12b","name":"NVIDIA Nemotron 3 Super 120B A12B","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":32000,"cost":{"input":0.15,"output":0.65}} +{"id":"nvidia/nemotron-3-ultra-550b-a55b","name":"Nemotron 3 Ultra","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"input":0.6,"output":2.4,"cache_read":0.12}} +{"id":"nvidia/nemotron-3.5-lightning","name":"Nemotron 3.5 Lightning 30B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"budget_tokens","min":1,"max":32768}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.05,"output":0.2,"cache_read":0.01}} +{"id":"nvidia/nemotron-nano-12b-v2-vl","name":"Nvidia Nemotron Nano 12B V2 VL","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.2,"output":0.6}} +{"id":"nvidia/nemotron-nano-9b-v2","name":"Nvidia Nemotron Nano 9B V2","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.06,"output":0.23}} +{"id":"openai/gpt-3.5-turbo","name":"GPT-3.5 Turbo","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":16385,"maxOutputTokens":4096,"cost":{"input":0.5,"output":1.5}} +{"id":"openai/gpt-4-turbo","name":"GPT-4 Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"input":10,"output":30},"status":"deprecated"} +{"id":"openai/gpt-4.1","name":"GPT-4.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"openai/gpt-4.1-fast","name":"GPT-4.1 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":3.5,"output":14,"cache_read":0.875}} +{"id":"openai/gpt-4.1-mini","name":"GPT-4.1 mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.4,"output":1.6,"cache_read":0.1}} +{"id":"openai/gpt-4.1-mini-fast","name":"GPT-4.1 mini (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.7,"output":2.8,"cache_read":0.175}} +{"id":"openai/gpt-4.1-nano","name":"GPT-4.1 nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.1,"output":0.4,"cache_read":0.025},"status":"deprecated"} +{"id":"openai/gpt-4.1-nano-fast","name":"GPT-4.1 nano (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"input":0.2,"output":0.8,"cache_read":0.05}} +{"id":"openai/gpt-4o","name":"GPT-4o","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":2.5,"output":10,"cache_read":1.25}} +{"id":"openai/gpt-4o-fast","name":"GPT-4o (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":4.25,"output":17,"cache_read":2.125}} +{"id":"openai/gpt-4o-mini","name":"GPT-4o mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.15,"output":0.6,"cache_read":0.075}} +{"id":"openai/gpt-4o-mini-fast","name":"GPT-4o mini (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"input":0.25,"output":1,"cache_read":0.125}} +{"id":"openai/gpt-5","name":"GPT-5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"openai/gpt-5-codex","name":"GPT-5-Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.13}} +{"id":"openai/gpt-5-fast","name":"GPT-5 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":20,"cache_read":0.25}} +{"id":"openai/gpt-5-mini","name":"GPT-5 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.025}} +{"id":"openai/gpt-5-mini-fast","name":"GPT-5 mini (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.45,"output":3.6,"cache_read":0.045}} +{"id":"openai/gpt-5-nano","name":"GPT-5 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.05,"output":0.4,"cache_read":0.005}} +{"id":"openai/gpt-5-pro","name":"GPT-5 pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high"]}],"contextWindow":400000,"maxOutputTokens":272000,"cost":{"input":15,"output":120}} +{"id":"openai/gpt-5.1-codex","name":"GPT-5.1-Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.13}} +{"id":"openai/gpt-5.1-codex-max","name":"GPT 5.1 Codex Max","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"openai/gpt-5.1-codex-mini","name":"GPT-5.1 Codex mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.25,"output":2,"cache_read":0.03}} +{"id":"openai/gpt-5.1-thinking","name":"GPT 5.1 Thinking","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.25,"output":10,"cache_read":0.125}} +{"id":"openai/gpt-5.1-thinking-fast","name":"GPT 5.1 Thinking (Fast)","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":20,"cache_read":0.25}} +{"id":"openai/gpt-5.2","name":"GPT-5.2","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"openai/gpt-5.2-codex","name":"GPT-5.2-Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"openai/gpt-5.2-fast","name":"GPT 5.2 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":3.5,"output":28,"cache_read":0.35}} +{"id":"openai/gpt-5.2-pro","name":"GPT 5.2 ","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":21,"output":168}} +{"id":"openai/gpt-5.3-codex","name":"GPT 5.3 Codex","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.75,"output":14,"cache_read":0.175}} +{"id":"openai/gpt-5.3-codex-fast","name":"GPT 5.3 Codex (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":3.5,"output":28,"cache_read":0.35}} +{"id":"openai/gpt-5.4","name":"GPT 5.4","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2.5,"output":15,"cache_read":0.25}} +{"id":"openai/gpt-5.4-fast","name":"GPT 5.4 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5}} +{"id":"openai/gpt-5.4-mini","name":"GPT 5.4 Mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.75,"output":4.5,"cache_read":0.075}} +{"id":"openai/gpt-5.4-mini-fast","name":"GPT 5.4 Mini (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":1.5,"output":9,"cache_read":0.15}} +{"id":"openai/gpt-5.4-nano","name":"GPT 5.4 Nano","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":400000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.25,"cache_read":0.02}} +{"id":"openai/gpt-5.4-pro","name":"GPT 5.4 Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":30,"output":180}} +{"id":"openai/gpt-5.5","name":"GPT 5.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":5,"output":30,"cache_read":0.5}} +{"id":"openai/gpt-5.5-fast","name":"GPT 5.5 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":12.5,"output":75,"cache_read":1.25}} +{"id":"openai/gpt-5.5-pro","name":"GPT 5.5 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":30,"output":180}} +{"id":"openai/gpt-5.6-luna","name":"GPT 5.6 Luna","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.2,"output":1.2,"cache_read":0.02,"cache_write":0.25}} +{"id":"openai/gpt-5.6-luna-fast","name":"GPT 5.6 Luna (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":0.4,"output":2.4,"cache_read":0.04,"cache_write":0.5}} +{"id":"openai/gpt-5.6-sol","name":"GPT 5.6 Sol","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":10,"cache_read":0.2,"cache_write":2.5}} +{"id":"openai/gpt-5.6-sol-fast","name":"GPT 5.6 Sol (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":20,"cache_read":0.4,"cache_write":5}} +{"id":"openai/gpt-5.6-terra","name":"GPT 5.6 Terra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":2,"output":12,"cache_read":0.2,"cache_write":2.5}} +{"id":"openai/gpt-5.6-terra-fast","name":"GPT 5.6 Terra (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":4,"output":24,"cache_read":0.4,"cache_write":5}} +{"id":"openai/gpt-6-astra","name":"GPT-6 Astra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":10,"output":50,"cache_read":1,"cache_write":12.5,"tiers":[{"input":20,"output":75,"cache_read":2,"cache_write":25,"tier":{"type":"context","size":272001}}],"context_over_200k":{"input":20,"output":75,"cache_read":2,"cache_write":25}}} +{"id":"openai/gpt-6-astra-fast","name":"GPT-6 Astra (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high","xhigh","max"]}],"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"input":20,"output":100,"cache_read":2,"cache_write":25,"tiers":[{"input":40,"output":150,"cache_read":4,"cache_write":25,"tier":{"type":"context","size":272001}}],"context_over_200k":{"input":40,"output":150,"cache_read":4,"cache_write":25}}} +{"id":"openai/gpt-oss-120b","name":"GPT OSS 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":131072,"cost":{"input":0.1,"output":0.5}} +{"id":"openai/gpt-oss-20b","name":"GPT OSS 20B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":131072,"maxOutputTokens":8192,"cost":{"input":0.05,"output":0.2}} +{"id":"openai/gpt-oss-safeguard-120b","name":"GPT OSS Safeguard 120B","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16000,"cost":{"input":0.15,"output":0.6}} +{"id":"openai/gpt-oss-safeguard-20b","name":"gpt-oss-safeguard-20b","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":128000,"maxOutputTokens":16000,"cost":{"input":0.07,"output":0.2}} +{"id":"openai/gpt-realtime-2.1","name":"gpt-realtime-2.1","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high","xhigh"]}],"contextWindow":128000,"maxOutputTokens":32000,"cost":{"input":4,"output":24,"cache_read":0.4}} +{"id":"openai/o1","name":"o1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":15,"output":60,"cache_read":7.5},"status":"deprecated"} +{"id":"openai/o3","name":"o3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"openai/o3-fast","name":"o3 (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":3.5,"output":14,"cache_read":0.875}} +{"id":"openai/o3-mini","name":"o3-mini","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.55},"status":"deprecated"} +{"id":"openai/o3-pro","name":"o3 Pro","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":20,"output":80}} +{"id":"openai/o4-mini","name":"o4-mini","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":1.1,"output":4.4,"cache_read":0.275},"status":"deprecated"} +{"id":"openai/o4-mini-fast","name":"o4-mini (Fast)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":200000,"maxOutputTokens":100000,"cost":{"input":2,"output":8,"cache_read":0.5}} +{"id":"openai/text-embedding-3-large","name":"text-embedding-3-large","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"openai/text-embedding-3-small","name":"text-embedding-3-small","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"openai/text-embedding-ada-002","name":"text-embedding-ada-002","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"perplexity/sonar","name":"Sonar","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":127000,"maxOutputTokens":8000} +{"id":"perplexity/sonar-pro","name":"Sonar Pro","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":200000,"maxOutputTokens":8000} +{"id":"perplexity/sonar-reasoning-pro","name":"Sonar Reasoning Pro","toolCall":false,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["minimal","low","medium","high"]}],"contextWindow":127000,"maxOutputTokens":8000} +{"id":"poolside/laguna-s-2.1","name":"Laguna S 2.1","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.1,"output":0.2,"cache_read":0.01}} +{"id":"poolside/laguna-s-2.1-free","name":"Laguna S 2.1 Free","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":256000,"maxOutputTokens":32768,"cost":{"input":0,"output":0}} +{"id":"sakana/fugu-ultra","name":"Fugu Ultra","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":5,"output":30,"cache_read":0.5}} +{"id":"sakana/namazu","name":"Sakana Namazu","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.95,"output":4,"cache_read":0.15}} +{"id":"spacexai/grok-4.1-fast-non-reasoning","name":"Grok 4.1 Fast Non-Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":0.2,"output":0.5,"cache_read":0.05}} +{"id":"spacexai/grok-4.1-fast-reasoning","name":"Grok 4.1 Fast Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":0.2,"output":0.5,"cache_read":0.05}} +{"id":"spacexai/grok-4.20-multi-agent","name":"Grok 4.20 Multi-Agent","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.20-multi-agent-beta","name":"Grok 4.20 Multi Agent Beta","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.20-non-reasoning","name":"Grok 4.20 Non-Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.20-non-reasoning-beta","name":"Grok 4.20 Beta Non-Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":false,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.4}} +{"id":"spacexai/grok-4.20-reasoning","name":"Grok 4.20 Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.20-reasoning-beta","name":"Grok 4.20 Beta Reasoning","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.3","name":"Grok 4.3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2}} +{"id":"spacexai/grok-4.5","name":"Grok 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.3}} +{"id":"spacexai/grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.5}} +{"id":"spacexai/grok-build-0.1","name":"Grok Build 0.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":1,"output":2,"cache_read":0.2}} +{"id":"stepfun/step-3.5-flash","name":"StepFun 3.5 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":262114,"maxOutputTokens":262114,"cost":{"input":0.09,"output":0.3,"cache_read":0.02}} +{"id":"stepfun/step-3.7-flash","name":"Step 3.7 Flash","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":0.2,"output":1.15,"cache_read":0.04}} +{"id":"tencent/hy-mt2-lite","name":"Tencent Hy-MT2-Lite","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8000,"maxOutputTokens":4000,"cost":{"input":0.044,"output":0.177}} +{"id":"tencent/hy-mt2-plus","name":"Tencent Hy-MT2-Plus","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8000,"maxOutputTokens":4000,"cost":{"input":0.074,"output":0.295}} +{"id":"tencent/hy-mt2-pro","name":"Tencent Hy-MT2-Pro","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8000,"maxOutputTokens":4000,"cost":{"input":0.074,"output":0.295}} +{"id":"tencent/hy3","name":"Hy3","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","high"]}],"contextWindow":262144,"maxOutputTokens":262144,"cost":{"input":0.14,"output":0.58,"cache_read":0.035}} +{"id":"tencent/hy4-preview","name":"Tencent Hy4 Preview","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","high"]}],"contextWindow":1024000,"maxOutputTokens":64000,"cost":{"input":0.834,"output":2.501,"cache_read":0.042}} +{"id":"thinkingmachines/inkling","name":"Inkling","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":1,"output":4.05,"cache_read":0.17}} +{"id":"thinkingmachines/inkling-small","name":"Inkling Small","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","minimal","low","medium","high","xhigh","max"]}],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":0.5,"output":1.2,"cache_read":0.1}} +{"id":"voyage/rerank-2.5","name":"Voyage Rerank 2.5","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000} +{"id":"voyage/rerank-2.5-lite","name":"Voyage Rerank 2.5 Lite","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000} +{"id":"voyage/voyage-3-large","name":"voyage-3-large","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-3.5","name":"voyage-3.5","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-3.5-lite","name":"voyage-3.5-lite","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-code-2","name":"voyage-code-2","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-code-3","name":"voyage-code-3","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-finance-2","name":"voyage-finance-2","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"voyage/voyage-law-2","name":"voyage-law-2","toolCall":false,"structuredOutput":false,"imageInput":false,"reasoning":false,"contextWindow":8192,"maxOutputTokens":1536} +{"id":"xiaomi/mimo-v2.5","name":"MiMo M2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1050000,"maxOutputTokens":131100,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028}} +{"id":"xiaomi/mimo-v2.5-pro","name":"MiMo V2.5 Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1050000,"maxOutputTokens":131000,"cost":{"input":0.435,"output":0.87,"cache_read":0.0036}} +{"id":"xiaomi/mimo-v2.5-pro-ultraspeed","name":"MiMo V2.5 Pro UltraSpeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1.305,"output":2.61,"cache_read":0.0108}} +{"id":"zai/glm-4.5","name":"GLM 4.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":128000,"maxOutputTokens":96000,"cost":{"input":0.6,"output":2.2,"cache_read":0.11}} +{"id":"zai/glm-4.5-air","name":"GLM 4.5 Air","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":128000,"maxOutputTokens":96000,"cost":{"input":0.2,"output":1.1,"cache_read":0.03}} +{"id":"zai/glm-4.5v","name":"GLM 4.5V","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":66000,"maxOutputTokens":16000,"cost":{"input":0.6,"output":1.8,"cache_read":0.11}} +{"id":"zai/glm-4.6","name":"GLM 4.6","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":96000,"cost":{"input":0.6,"output":2.2,"cache_read":0.11}} +{"id":"zai/glm-4.7","name":"GLM 4.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":120000,"cost":{"input":0.6,"output":2.2,"cache_read":0.12}} +{"id":"zai/glm-4.7-flash","name":"GLM 4.7 Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131000,"cost":{"input":0.07,"output":0.4}} +{"id":"zai/glm-4.7-flashx","name":"GLM 4.7 FlashX","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":128000,"cost":{"input":0.06,"output":0.4,"cache_read":0.01}} +{"id":"zai/glm-5","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":131100,"cost":{"input":1,"output":3.2}} +{"id":"zai/glm-5-turbo","name":"GLM 5 Turbo","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":131100,"cost":{"input":1.2,"output":4,"cache_read":0.24}} +{"id":"zai/glm-5.1","name":"GLM 5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":202800,"maxOutputTokens":64000,"cost":{"input":1.4,"output":4.4,"cache_read":0.26}} +{"id":"zai/glm-5.2","name":"GLM 5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":0.8,"output":2.55,"cache_read":0.16}} +{"id":"zai/glm-5.2-fast","name":"GLM 5.2 Fast","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"},{"type":"effort","values":["high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"input":2.1,"output":6.6,"cache_read":0.21}} +{"id":"zai/glm-5.3","name":"GLM 5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"input":0.7,"output":2.2,"cache_read":0.13}} +{"id":"zai/glm-5.3-fast","name":"GLM 5.3 Fast","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"input":2.1,"output":6.6,"cache_read":0.21}} +{"id":"zai/glm-5.3-flash","name":"GLM 5.3 Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131000,"cost":{"input":0.15,"output":0.5,"cache_read":0.03}} +{"id":"zai/glm-5.3-promo-50","name":"GLM 5.3 (50% off)","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[],"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"input":0.7,"output":2.2,"cache_read":0.13}} +{"id":"zai/glm-5v-turbo","name":"GLM 5V Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":128000,"cost":{"input":1.2,"output":4,"cache_read":0.24}} diff --git a/packages/ai/catalog/sources/models-dev/providers/xai.jsonl b/packages/ai/catalog/sources/models-dev/providers/xai.jsonl new file mode 100644 index 0000000..14979bc --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/xai.jsonl @@ -0,0 +1,7 @@ +{"id":"grok-4.20-0309-non-reasoning","name":"Grok 4.20 (Non-Reasoning)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":false,"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2,"tiers":[{"input":2.5,"output":5,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":5,"cache_read":0.4}}} +{"id":"grok-4.20-0309-reasoning","name":"Grok 4.20 (Reasoning)","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2,"tiers":[{"input":2.5,"output":5,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":5,"cache_read":0.4}}} +{"id":"grok-4.20-multi-agent-0309","name":"Grok 4.20 Multi-Agent","toolCall":false,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2,"tiers":[{"input":2.5,"output":5,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":5,"cache_read":0.4}}} +{"id":"grok-4.3","name":"Grok 4.3","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["none","low","medium","high"]}],"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"input":1.25,"output":2.5,"cache_read":0.2,"tiers":[{"input":2.5,"output":5,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2.5,"output":5,"cache_read":0.4}}} +{"id":"grok-4.5","name":"Grok 4.5","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.3,"tiers":[{"input":4,"output":12,"cache_read":0.6,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":0.6}}} +{"id":"grok-4.6","name":"Grok 4.6","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","medium","high","xhigh"]}],"contextWindow":500000,"maxOutputTokens":500000,"cost":{"input":2,"output":6,"cache_read":0.5,"tiers":[{"input":4,"output":12,"cache_read":1,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":4,"output":12,"cache_read":1}}} +{"id":"grok-build-0.1","name":"Grok Build 0.1","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[],"contextWindow":256000,"maxOutputTokens":256000,"cost":{"input":1,"output":2,"cache_read":0.2,"tiers":[{"input":2,"output":4,"cache_read":0.4,"tier":{"type":"context","size":200000}}],"context_over_200k":{"input":2,"output":4,"cache_read":0.4}}} diff --git a/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-ams.jsonl b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-ams.jsonl new file mode 100644 index 0000000..d93015e --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-ams.jsonl @@ -0,0 +1,3 @@ +{"id":"mimo-v2-pro","name":"MiMo-V2-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2.5","name":"MiMo-V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"mimo-v2.5-pro","name":"MiMo-V2.5-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-cn.jsonl b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-cn.jsonl new file mode 100644 index 0000000..d93015e --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-cn.jsonl @@ -0,0 +1,3 @@ +{"id":"mimo-v2-pro","name":"MiMo-V2-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2.5","name":"MiMo-V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"mimo-v2.5-pro","name":"MiMo-V2.5-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-sgp.jsonl b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-sgp.jsonl new file mode 100644 index 0000000..d93015e --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/xiaomi-token-plan-sgp.jsonl @@ -0,0 +1,3 @@ +{"id":"mimo-v2-pro","name":"MiMo-V2-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0},"status":"deprecated"} +{"id":"mimo-v2.5","name":"MiMo-V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} +{"id":"mimo-v2.5-pro","name":"MiMo-V2.5-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/xiaomi.jsonl b/packages/ai/catalog/sources/models-dev/providers/xiaomi.jsonl new file mode 100644 index 0000000..7d70518 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/xiaomi.jsonl @@ -0,0 +1,6 @@ +{"id":"mimo-v2-flash","name":"MiMo-V2-Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":65536,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028},"status":"deprecated"} +{"id":"mimo-v2-omni","name":"MiMo-V2-Omni","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":262144,"maxOutputTokens":131072,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028},"status":"deprecated"} +{"id":"mimo-v2-pro","name":"MiMo-V2-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.435,"output":0.87,"cache_read":0.0036},"status":"deprecated"} +{"id":"mimo-v2.5","name":"MiMo-V2.5","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.14,"output":0.28,"cache_read":0.0028}} +{"id":"mimo-v2.5-pro","name":"MiMo-V2.5-Pro","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":0.435,"output":0.87,"cache_read":0.0036}} +{"id":"mimo-v2.5-pro-ultraspeed","name":"MiMo-V2.5-Pro-UltraSpeed","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"input":1.305,"output":2.61,"cache_read":0.0108},"status":"beta"} diff --git a/packages/ai/catalog/sources/models-dev/providers/zai.jsonl b/packages/ai/catalog/sources/models-dev/providers/zai.jsonl new file mode 100644 index 0000000..27dd347 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/zai.jsonl @@ -0,0 +1,16 @@ +{"id":"glm-4.5","name":"GLM-4.5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":98304,"cost":{"input":0.6,"output":2.2,"cache_read":0.11,"cache_write":0}} +{"id":"glm-4.5-air","name":"GLM-4.5-Air","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":98304,"cost":{"input":0.2,"output":1.1,"cache_read":0.03,"cache_write":0}} +{"id":"glm-4.5-flash","name":"GLM-4.5-Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":131072,"maxOutputTokens":98304,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-4.5v","name":"GLM-4.5V","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":64000,"maxOutputTokens":16384,"cost":{"input":0.6,"output":1.8}} +{"id":"glm-4.6","name":"GLM-4.6","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2,"cache_read":0.11,"cache_write":0}} +{"id":"glm-4.6v","name":"GLM-4.6V","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":0.3,"output":0.9}} +{"id":"glm-4.7","name":"GLM-4.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0.6,"output":2.2,"cache_read":0.11,"cache_write":0}} +{"id":"glm-4.7-flash","name":"GLM-4.7-Flash","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-4.7-flashx","name":"GLM-4.7-FlashX","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0.07,"output":0.4,"cache_read":0.01,"cache_write":0}} +{"id":"glm-5","name":"GLM-5","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":1,"output":3.2,"cache_read":0.2,"cache_write":0}} +{"id":"glm-5-turbo","name":"GLM-5-Turbo","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":1.2,"output":4,"cache_read":0.24,"cache_write":0}} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26,"cache_write":0}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26,"cache_write":0}} +{"id":"glm-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":1.4,"output":4.4,"cache_read":0.26,"cache_write":0}} +{"id":"glm-5.3-flash","name":"GLM-5.3-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0.075,"output":0.25,"cache_read":0.015,"cache_write":0}} +{"id":"glm-5v-turbo","name":"GLM-5V-Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":1.2,"output":4,"cache_read":0.24,"cache_write":0}} diff --git a/packages/ai/catalog/sources/models-dev/providers/zhipuai-coding-plan.jsonl b/packages/ai/catalog/sources/models-dev/providers/zhipuai-coding-plan.jsonl new file mode 100644 index 0000000..1061074 --- /dev/null +++ b/packages/ai/catalog/sources/models-dev/providers/zhipuai-coding-plan.jsonl @@ -0,0 +1,10 @@ +{"id":"glm-4.6v","name":"GLM-4.6V","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":128000,"maxOutputTokens":32768,"cost":{"input":0.3,"output":0.9}} +{"id":"glm-4.7","name":"GLM-4.7","toolCall":true,"structuredOutput":false,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":204800,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5-turbo","name":"GLM-5-Turbo","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.1","name":"GLM-5.1","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.2","name":"GLM-5.2","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.2-highspeed","name":"GLM-5.2 Highspeed","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.3","name":"GLM-5.3","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.3-flash","name":"GLM-5.3-Flash","toolCall":true,"structuredOutput":true,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5.3-highspeed","name":"GLM-5.3 Highspeed","toolCall":true,"structuredOutput":true,"imageInput":false,"reasoning":true,"reasoningOptions":[{"type":"effort","values":["low","high","max"]}],"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} +{"id":"glm-5v-turbo","name":"GLM-5V-Turbo","toolCall":true,"structuredOutput":false,"imageInput":true,"reasoning":true,"reasoningOptions":[{"type":"toggle"}],"contextWindow":200000,"maxOutputTokens":131072,"cost":{"input":0,"output":0,"cache_read":0,"cache_write":0}} diff --git a/packages/ai/package.json b/packages/ai/package.json index 9e628d3..f698610 100644 --- a/packages/ai/package.json +++ b/packages/ai/package.json @@ -24,7 +24,13 @@ "typecheck": "tsc --noEmit" }, "dependencies": { + "@aws-sdk/credential-provider-node": "3.972.82", "@axl/protocol": "workspace:*", + "@azure/identity": "4.13.2", + "@smithy/hash-node": "4.5.2", + "@smithy/protocol-http": "5.6.2", + "@smithy/signature-v4": "5.7.3", + "google-auth-library": "11.0.2", "undici": "8.10.2" } } diff --git a/packages/ai/scripts/generate-catalog.ts b/packages/ai/scripts/generate-catalog.ts new file mode 100644 index 0000000..06681b2 --- /dev/null +++ b/packages/ai/scripts/generate-catalog.ts @@ -0,0 +1,314 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createHash } from "node:crypto"; +import { mkdirSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { dirname, relative, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { + normalizeCatalogModels, + object, + positiveInteger, + type SourceModel, + string, +} from "../src/catalog-normalization.ts"; +import { PROVIDER_CATALOG_OVERLAYS } from "../src/catalog-overlays.ts"; +import { validateEndpointPolicy, validateModelCatalog } from "../src/catalog-validation.ts"; +import type { ModelInfo } from "../src/model.ts"; + +interface SourceProvider { + readonly models?: unknown; +} + +interface SourceManifest { + readonly _provenance?: unknown; + readonly providers?: unknown; +} + +interface SourceProviderIndex { + readonly id?: unknown; + readonly name?: unknown; + readonly documentation?: unknown; + readonly file?: unknown; + readonly modelCount?: unknown; + readonly sha256?: unknown; +} + +interface SourceManifestIndex { + readonly schemaVersion?: unknown; + readonly _provenance?: unknown; + readonly providers?: unknown; +} + +export interface GeneratedCatalogFiles { + readonly files: ReadonlyMap; +} + +const PACKAGE_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), ".."); +const DEFAULT_TARGET = resolve(PACKAGE_ROOT, "src/catalog.generated.ts"); +const GENERATED_SHARD_DIRECTORY = resolve(PACKAGE_ROOT, "src/catalog.generated"); +const CHECK_FLAG = `-${"-"}check`; +const EXPECTED_PROVIDER_IDS = [ + "amazon-bedrock", + "ant-ling", + "anthropic", + "azure-openai-responses", + "baseten", + "cerebras", + "cloudflare-ai-gateway", + "cloudflare-workers-ai", + "custom", + "deepseek", + "fireworks", + "github-copilot", + "google", + "google-vertex", + "groq", + "huggingface", + "kimi-coding", + "minimax", + "minimax-cn", + "mistral", + "moonshotai", + "moonshotai-cn", + "nvidia", + "openai", + "openai-codex", + "opencode", + "opencode-go", + "openrouter", + "qwen-token-plan", + "qwen-token-plan-cn", + "qwen-token-plan-individual", + "radius", + "together", + "vercel-ai-gateway", + "xai", + "xiaomi", + "xiaomi-token-plan-ams", + "xiaomi-token-plan-cn", + "xiaomi-token-plan-sgp", + "zai", + "zai-coding-cn", +] as const; + +function readManifest(name: "models-dev" | "ant-ling"): SourceManifest { + const sourceRoot = resolve(PACKAGE_ROOT, `catalog/sources/${name}`); + const manifestText = readFileSync(resolve(sourceRoot, "manifest.json"), "utf8"); + const manifest = JSON.parse(manifestText) as SourceManifestIndex; + if (`${JSON.stringify(manifest, null, 2)}\n` !== manifestText) { + throw new Error(`${name} manifest is not canonical JSON`); + } + if (manifest.schemaVersion !== 1) throw new Error(`${name} has an unsupported schema version`); + if (!Array.isArray(manifest.providers)) throw new Error(`${name}.providers must be an array`); + + const providers = new Map(); + let previousProviderId: string | undefined; + const indexedFiles = new Set(); + for (const [providerIndex, value] of manifest.providers.entries()) { + const entry = object(value, `${name}.providers[${providerIndex}]`) as SourceProviderIndex; + const providerId = string(entry.id, `${name}.providers[${providerIndex}].id`); + if (previousProviderId !== undefined && previousProviderId.localeCompare(providerId) >= 0) { + throw new Error(`${name} provider index is not strictly ordered`); + } + previousProviderId = providerId; + string(entry.name, `${name}/${providerId}.name`); + string(entry.documentation, `${name}/${providerId}.documentation`); + const file = string(entry.file, `${name}/${providerId}.file`); + if (file !== `providers/${providerId}.jsonl`) { + throw new Error(`${name}/${providerId} has a noncanonical shard path`); + } + const shardText = readFileSync(resolve(sourceRoot, file), "utf8"); + if (!shardText.endsWith("\n") || shardText.includes("\n\n")) { + throw new Error(`${name}/${providerId} shard must contain one model per line`); + } + const digest = createHash("sha256").update(shardText).digest("hex"); + if (digest !== string(entry.sha256, `${name}/${providerId}.sha256`)) { + throw new Error(`${name}/${providerId} shard checksum does not match its index`); + } + const models = new Map(); + let previousModelId: string | undefined; + for (const [lineIndex, line] of shardText.trimEnd().split("\n").entries()) { + const model = object( + JSON.parse(line), + `${name}/${providerId}:${lineIndex + 1}`, + ) as SourceModel; + const modelId = string(model.id, `${name}/${providerId}:${lineIndex + 1}.id`); + if (JSON.stringify(model) !== line) { + throw new Error(`${name}/${providerId}:${lineIndex + 1} is not canonical JSON`); + } + if (previousModelId !== undefined && previousModelId.localeCompare(modelId) >= 0) { + throw new Error(`${name}/${providerId} models are not strictly ordered`); + } + previousModelId = modelId; + models.set(modelId, model); + } + if (models.size !== positiveInteger(entry.modelCount, `${name}/${providerId}.modelCount`)) { + throw new Error(`${name}/${providerId} model count does not match its index`); + } + providers.set(providerId, { models: Object.fromEntries(models) }); + indexedFiles.add(`${providerId}.jsonl`); + } + + const actualFiles = readdirSync(resolve(sourceRoot, "providers")).sort(); + const expectedFiles = [...indexedFiles].sort(); + if (JSON.stringify(actualFiles) !== JSON.stringify(expectedFiles)) { + throw new Error(`${name} provider shards do not exactly match the index`); + } + return { _provenance: manifest._provenance, providers: Object.fromEntries(providers) }; +} + +function sortedRecord(entries: Iterable): Record { + return Object.fromEntries([...entries].sort(([left], [right]) => left.localeCompare(right))); +} + +function validateOverlays(): void { + const ids = PROVIDER_CATALOG_OVERLAYS.map((overlay) => overlay.id).sort(); + if (JSON.stringify(ids) !== JSON.stringify([...EXPECTED_PROVIDER_IDS].sort())) { + throw new Error("Provider overlays do not exactly cover the planned provider identities"); + } + const regions = new Set(); + const endpoints = new Set(); + for (const overlay of PROVIDER_CATALOG_OVERLAYS) { + if (overlay.catalogKind === "static" && overlay.source === undefined) { + throw new Error(`Static provider ${overlay.id} has no source manifest`); + } + if (overlay.catalogKind !== "static" && overlay.source !== undefined) { + throw new Error(`${overlay.id} cannot use a static source manifest`); + } + if (overlay.endpoint !== undefined) validateEndpointPolicy(overlay.endpoint, overlay.id); + if ((overlay.regionFamily === undefined) !== (overlay.region === undefined)) { + throw new Error(`${overlay.id} must define both regional fields or neither`); + } + if (overlay.regionFamily !== undefined && overlay.region !== undefined) { + const regionKey = `${overlay.regionFamily}/${overlay.region}`; + if (regions.has(regionKey)) throw new Error(`Duplicate regional catalog ${regionKey}`); + regions.add(regionKey); + const endpoint = JSON.stringify(overlay.endpoint); + const endpointKey = `${overlay.regionFamily}/${endpoint}`; + if (endpoints.has(endpointKey)) { + throw new Error(`${overlay.regionFamily} regional catalogs share an endpoint`); + } + endpoints.add(endpointKey); + } + } +} + +const GENERATED_HEADER = `// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. +`; + +function modelConstant(providerId: string): string { + return `${providerId.replaceAll(/[^a-zA-Z0-9]/g, "_").toUpperCase()}_MODELS`; +} + +function lines(values: readonly T[]): string { + return values.map((value) => ` ${JSON.stringify(value)},`).join("\n"); +} + +export function generateCatalog(): GeneratedCatalogFiles { + validateOverlays(); + const manifests = { + "models-dev": readManifest("models-dev"), + "ant-ling": readManifest("ant-ling"), + }; + const catalogEntries: [string, readonly ModelInfo[]][] = []; + for (const overlay of PROVIDER_CATALOG_OVERLAYS) { + if (overlay.catalogKind !== "static" || overlay.source === undefined) continue; + const manifestProviders = object( + manifests[overlay.source.manifest].providers, + `${overlay.source.manifest}.providers`, + ); + const sourceProvider = object( + manifestProviders[overlay.source.providerId], + `${overlay.source.manifest}/${overlay.source.providerId}`, + ) as SourceProvider; + const models = object( + sourceProvider.models, + `${overlay.source.manifest}/${overlay.source.providerId}.models`, + ); + const normalized = normalizeCatalogModels(overlay, models); + catalogEntries.push([overlay.id, normalized]); + } + const staticCatalog = sortedRecord(catalogEntries); + validateModelCatalog(Object.values(staticCatalog).flat()); + + const modelsDevProvenance = object(manifests["models-dev"]._provenance, "models-dev provenance"); + const antProvenance = object(manifests["ant-ling"]._provenance, "ant-ling provenance"); + const provenance = { + generatedAt: string(modelsDevProvenance.retrievedAt, "models-dev retrievedAt"), + sources: [ + { + name: "models.dev", + location: string(modelsDevProvenance.source, "models-dev source"), + retrievedAt: string(modelsDevProvenance.retrievedAt, "models-dev retrievedAt"), + sha256: string(modelsDevProvenance.sourceSha256, "models-dev sourceSha256"), + revision: string(modelsDevProvenance.repositoryCommit, "models-dev repositoryCommit"), + license: "MIT", + }, + { + name: "Ant Ling official documentation", + location: string((antProvenance.sources as unknown[] | undefined)?.[0], "ant-ling source"), + retrievedAt: string(antProvenance.retrievedAt, "ant-ling retrievedAt"), + sha256: string(antProvenance.sourceSha256, "ant-ling sourceSha256"), + license: "factual metadata", + }, + ], + }; + const providers = PROVIDER_CATALOG_OVERLAYS.map( + ({ id, displayName, catalogKind, regionFamily, region }) => ({ + id, + displayName, + catalogKind, + ...(regionFamily === undefined ? {} : { regionFamily }), + ...(region === undefined ? {} : { region }), + }), + ).sort((left, right) => left.id.localeCompare(right.id)); + + const files = new Map(); + const imports: string[] = []; + const catalogProperties: string[] = []; + for (const [providerId, models] of Object.entries(staticCatalog)) { + const constant = modelConstant(providerId); + const target = resolve(GENERATED_SHARD_DIRECTORY, `${providerId}.generated.ts`); + imports.push( + `import { MODELS as ${constant} } from "./catalog.generated/${providerId}.generated.ts";`, + ); + catalogProperties.push(` ${JSON.stringify(providerId)}: ${constant},`); + files.set( + target, + `${GENERATED_HEADER}\nimport type { ModelInfo } from "../model.ts";\n\nexport const MODELS: readonly ModelInfo[] = [\n${lines(models)}\n];\n`, + ); + } + + files.set( + DEFAULT_TARGET, + `${GENERATED_HEADER}\nimport type { BuiltinCatalogProvider, CatalogProvenance } from "./catalog.ts";\nimport type { ModelInfo } from "./model.ts";\n${imports.join("\n")}\n\nexport const GENERATED_CATALOG_PROVENANCE = ${JSON.stringify(provenance)} as const satisfies CatalogProvenance;\n\nexport const BUILTIN_CATALOG_PROVIDERS = [\n${lines(providers)}\n] as const satisfies readonly BuiltinCatalogProvider[];\n\nexport const STATIC_MODEL_CATALOG: Readonly> = {\n${catalogProperties.join("\n")}\n};\n`, + ); + return { files }; +} + +const invokedPath = process.argv[1] ? resolve(process.argv[1]) : ""; +if (invokedPath === fileURLToPath(import.meta.url)) { + const check = process.argv[2] === CHECK_FLAG; + const target = check ? resolve(process.cwd(), process.argv[3] ?? DEFAULT_TARGET) : undefined; + const output = generateCatalog(); + if (check) { + const expected = target === undefined ? undefined : output.files.get(target); + if (expected === undefined || target === undefined) { + throw new Error( + `No generated catalog output exists for ${relative(process.cwd(), target ?? "")}`, + ); + } + const filesToCheck = target === DEFAULT_TARGET ? output.files : new Map([[target, expected]]); + for (const [path, source] of filesToCheck) { + if (readFileSync(path, "utf8") !== source) process.exitCode = 1; + } + } else { + rmSync(GENERATED_SHARD_DIRECTORY, { recursive: true, force: true }); + mkdirSync(GENERATED_SHARD_DIRECTORY, { recursive: true }); + for (const [path, source] of output.files) writeFileSync(path, source); + } +} diff --git a/packages/ai/src/ant-ling.ts b/packages/ai/src/ant-ling.ts new file mode 100644 index 0000000..fab8879 --- /dev/null +++ b/packages/ai/src/ant-ling.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const ANT_LING_PROVIDER_ID = "ant-ling"; +export const ANT_LING_API_KEY_ENV = "ANT_LING_API_KEY"; +export const ANT_LING_BASE_URL = "https://api.ant-ling.com/v1"; + +export const ANT_LING_PROVIDER_DEFINITION = { + id: ANT_LING_PROVIDER_ID, + displayName: "Ant Ling", + apiKeyDisplayName: "Ant Ling API key", + environmentVariables: [ANT_LING_API_KEY_ENV], + baseUrl: ANT_LING_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createAntLingProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(ANT_LING_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/anthropic-messages.ts b/packages/ai/src/anthropic-messages.ts new file mode 100644 index 0000000..1600502 --- /dev/null +++ b/packages/ai/src/anthropic-messages.ts @@ -0,0 +1,895 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Axl-native Anthropic Messages request and streaming response codec. + +import type { JsonObject, JsonValue, Usage } from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { AnthropicCompatibility, ModelInfo, ModelStreamEvent } from "./model.ts"; +import { + isPreparedModelRequest, + type CachePlacement, + type PreparedModelRequest, + type PreparedRequestMessage, +} from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; +import { modelCostRates, withUsageCost } from "./usage.ts"; + +const ANTHROPIC_VERSION = "2023-06-01"; +const INTERLEAVED_THINKING_BETA = "interleaved-thinking-2025-05-14"; +const RESERVED_REQUEST_FIELDS = new Set([ + "model", + "messages", + "system", + "max_tokens", + "stream", + "tools", + "tool_choice", + "thinking", + "output_config", + "temperature", + "top_p", + "top_k", + "metadata", +]); + +export class AnthropicMessagesCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "AnthropicMessagesCodecError"; + } +} + +export interface EncodedAnthropicMessagesRequest { + readonly body: JsonObject; + /** Protocol headers only. Authentication remains transport-owned. */ + readonly headers: Readonly>; +} + +type MutableJsonObject = Record; + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function compatibility(model: ModelInfo): AnthropicCompatibility { + if (model.apiDialect !== "anthropic-messages") { + throw new AnthropicMessagesCodecError( + `Model ${model.modelId} does not use the anthropic-messages dialect`, + ); + } + if (model.compatibility?.dialect !== "anthropic-messages") { + throw new AnthropicMessagesCodecError( + `Model ${model.modelId} has no Anthropic Messages compatibility record`, + ); + } + return model.compatibility; +} + +function preparedRequest(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new AnthropicMessagesCodecError("Anthropic Messages requires a prepared model request"); + } +} + +function hasPlacement( + placements: readonly CachePlacement[], + target: CachePlacement["target"], + messageIndex?: number, + contentIndex?: number, + toolIndex?: number, +): boolean { + return placements.some( + (placement) => + placement.target === target && + placement.messageIndex === messageIndex && + placement.contentIndex === contentIndex && + placement.toolIndex === toolIndex, + ); +} + +function cacheControl( + model: ModelInfo, + request: PreparedModelRequest, +): MutableJsonObject | undefined { + const retention = request.preparation.cache.retention; + if (retention === "none") return undefined; + if (retention === "long" && compatibility(model).supportsLongCacheRetention !== true) { + throw new AnthropicMessagesCodecError("Long cache retention is unsupported by this model"); + } + return retention === "long" ? { type: "ephemeral", ttl: "1h" } : { type: "ephemeral" }; +} + +function imageBlock(request: PreparedModelRequest, sha256: string): MutableJsonObject { + const blob = request.preparation.blobs.get(sha256); + if (blob === undefined) { + throw new AnthropicMessagesCodecError(`Prepared blob ${sha256} is unavailable`); + } + if ( + !new Set(["image/jpeg", "image/png", "image/gif", "image/webp"]).has(blob.reference.mediaType) + ) { + throw new AnthropicMessagesCodecError( + `Anthropic Messages does not support image media type ${blob.reference.mediaType}`, + ); + } + return { + type: "image", + source: { + type: "base64", + media_type: blob.reference.mediaType, + data: Buffer.from(blob.bytes).toString("base64"), + }, + }; +} + +function basicContent( + request: PreparedModelRequest, + message: PreparedRequestMessage, + messageIndex: number, + marker: MutableJsonObject | undefined, +): JsonValue[] { + return message.content.map((content, contentIndex): JsonValue => { + const marked = + marker !== undefined && + hasPlacement( + request.preparation.cache.placements, + "message-content", + messageIndex, + contentIndex, + ); + const block = + content.type === "text" + ? ({ type: "text", text: content.text } as MutableJsonObject) + : content.type === "blob" + ? imageBlock(request, content.blob.sha256) + : undefined; + if (block === undefined) { + throw new AnthropicMessagesCodecError("Anthropic Messages cannot encode this content block"); + } + if (marked) block.cache_control = marker; + return block; + }); +} + +function rejectContinuation(message: PreparedRequestMessage, messageIndex: number): void { + if (message.role !== "assistant") return; + if (message.continuation !== undefined) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}] has continuation metadata unsupported by Anthropic Messages`, + ); + } + for (const [contentIndex, content] of message.content.entries()) { + if (content.type === "text" && content.continuation !== undefined) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported continuation metadata`, + ); + } + } + for (const [callIndex, call] of (message.toolCalls ?? []).entries()) { + if (call.continuation !== undefined) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported continuation metadata`, + ); + } + if (call.signature !== undefined) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported signature metadata`, + ); + } + } +} + +function assistantContent( + message: Extract, + messageIndex: number, +): JsonValue[] { + rejectContinuation(message, messageIndex); + const blocks: JsonValue[] = []; + for (const content of message.content) { + if (content.type === "blob") { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}] cannot replay an assistant image through Anthropic Messages`, + ); + } + if (content.type === "text") { + if (content.text.length > 0) blocks.push({ type: "text", text: content.text }); + continue; + } + if (content.redacted === true) { + if (content.signature === undefined) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}] has redacted thinking without a signature`, + ); + } + blocks.push({ type: "redacted_thinking", data: content.signature.value }); + continue; + } + if (content.signature === undefined) { + if (content.text.length > 0) blocks.push({ type: "text", text: content.text }); + continue; + } + blocks.push({ + type: "thinking", + thinking: content.text, + signature: content.signature.value, + }); + } + for (const call of message.toolCalls ?? []) { + blocks.push({ type: "tool_use", id: call.callId, name: call.name, input: call.input }); + } + if (blocks.length === 0) { + throw new AnthropicMessagesCodecError( + `messages[${messageIndex}] has no Anthropic-renderable assistant content`, + ); + } + return blocks; +} + +function encodeMessages( + request: PreparedModelRequest, + marker: MutableJsonObject | undefined, +): JsonValue[] { + const messages: MutableJsonObject[] = []; + for (let messageIndex = 0; messageIndex < request.messages.length; messageIndex += 1) { + const message = request.messages[messageIndex]; + if (message === undefined) continue; + if (message.role === "user") { + messages.push({ + role: "user", + content: basicContent(request, message, messageIndex, marker), + }); + continue; + } + if (message.role === "assistant") { + messages.push({ role: "assistant", content: assistantContent(message, messageIndex) }); + continue; + } + + const results: JsonValue[] = []; + let resultIndex = messageIndex; + while (resultIndex < request.messages.length) { + const result = request.messages[resultIndex]; + if (result?.role !== "tool") break; + const content = basicContent(request, result, resultIndex, marker); + results.push({ + type: "tool_result", + tool_use_id: result.callId, + content, + is_error: result.isError, + }); + resultIndex += 1; + } + messages.push({ role: "user", content: results }); + messageIndex = resultIndex - 1; + } + return messages; +} + +function encodeTools( + request: PreparedModelRequest, + marker: MutableJsonObject | undefined, +): JsonValue[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + return request.tools.map((tool, toolIndex): JsonValue => { + if (tool.preparedConstraint?.type === "grammar") { + throw new AnthropicMessagesCodecError( + `Anthropic Messages cannot render grammar-constrained tool ${tool.canonicalName}`, + ); + } + const rendered: MutableJsonObject = { + name: tool.name, + description: tool.description, + input_schema: tool.inputSchema, + ...(tool.preparedConstraint?.type === "json-schema" && tool.preparedConstraint.strict + ? { strict: true } + : {}), + }; + if ( + marker !== undefined && + hasPlacement(request.preparation.cache.placements, "tool", undefined, undefined, toolIndex) + ) { + rendered.cache_control = marker; + } + return rendered; + }); +} + +function applyThinking( + body: MutableJsonObject, + model: ModelInfo, + request: PreparedModelRequest, +): boolean { + const reasoning = request.preparation.reasoning; + if (reasoning === undefined) return false; + if (reasoning.effective === "off") { + body.thinking = { type: "disabled" }; + return false; + } + const compat = compatibility(model); + if (compat.forceAdaptiveThinking === true) { + body.thinking = { type: "adaptive", display: "summarized" }; + body.output_config = { effort: reasoning.providerValue ?? reasoning.effective }; + return true; + } + if (reasoning.tokenBudget === undefined) { + throw new AnthropicMessagesCodecError( + "Anthropic budget-based thinking requires a prepared token budget", + ); + } + body.thinking = { + type: "enabled", + budget_tokens: reasoning.tokenBudget, + display: "summarized", + }; + return true; +} + +function applySampling(body: MutableJsonObject, request: PreparedModelRequest): void { + const sampling = request.sampling; + if (sampling === undefined) return; + if (sampling.temperature !== undefined) body.temperature = sampling.temperature; + if (sampling.topP !== undefined) body.top_p = sampling.topP; + if (sampling.topK !== undefined) body.top_k = sampling.topK; + for (const [field, value] of Object.entries(sampling.custom ?? {})) { + if (RESERVED_REQUEST_FIELDS.has(field) || field in body) { + throw new AnthropicMessagesCodecError( + `Custom sampling field ${field} collides with an Anthropic request field`, + ); + } + body[field] = value; + } +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeAnthropicMessagesRequest( + model: ModelInfo, + request: PreparedModelRequest, +): EncodedAnthropicMessagesRequest { + preparedRequest(request); + const compat = compatibility(model); + if (request.modelId !== model.modelId) { + throw new AnthropicMessagesCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + const metadata = request.metadata; + if (metadata !== undefined && Object.keys(metadata).some((key) => key !== "user_id")) { + throw new AnthropicMessagesCodecError("Anthropic Messages supports only metadata.user_id"); + } + if (metadata?.user_id !== undefined && typeof metadata.user_id !== "string") { + throw new AnthropicMessagesCodecError("Anthropic Messages metadata.user_id must be a string"); + } + + const marker = cacheControl(model, request); + const body: MutableJsonObject = { + model: model.modelId, + messages: encodeMessages(request, marker), + max_tokens: request.maxOutputTokens ?? model.maxOutputTokens, + stream: true, + }; + if (request.system !== undefined && request.system.length > 0) { + body.system = [ + { + type: "text", + text: request.system, + ...(marker !== undefined && hasPlacement(request.preparation.cache.placements, "system") + ? { cache_control: marker } + : {}), + }, + ]; + } + const tools = encodeTools(request, marker); + if (tools !== undefined) body.tools = tools; + if (request.toolChoice !== undefined) { + if (request.toolChoice !== "none" && tools === undefined) { + throw new AnthropicMessagesCodecError( + `toolChoice ${request.toolChoice} needs at least one tool`, + ); + } + body.tool_choice = { + type: request.toolChoice === "required" ? "any" : request.toolChoice, + }; + } + const thinkingEnabled = applyThinking(body, model, request); + applySampling(body, request); + if (metadata?.user_id !== undefined) body.metadata = { user_id: metadata.user_id }; + + const headers: Record = { + accept: "text/event-stream", + "content-type": "application/json", + "anthropic-version": ANTHROPIC_VERSION, + }; + if (thinkingEnabled && compat.forceAdaptiveThinking !== true) { + headers["anthropic-beta"] = INTERLEAVED_THINKING_BETA; + } + return { body, headers }; +} + +interface TextBlock { + readonly type: "text"; + readonly providerIndex: number; + readonly contentIndex: number; +} + +interface ThinkingBlock { + readonly type: "thinking"; + readonly providerIndex: number; + readonly contentIndex: number; + readonly redacted: boolean; + signature: string; +} + +interface ToolBlock { + readonly type: "tool"; + readonly providerIndex: number; + readonly contentIndex: number; + readonly callId: string; + readonly wireName: string; + argumentsText: string; +} + +type Block = TextBlock | ThinkingBlock | ToolBlock; + +function numberField(value: unknown, name: string): number | undefined { + if (value === undefined || value === null) return undefined; + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new AnthropicMessagesCodecError(`Anthropic usage ${name} must be non-negative`); + } + return value; +} + +interface AnthropicUsageState { + input: number; + output: number; + cacheRead: number; + cacheWrite: number; + cacheWrite1h: number; + reasoning: number; +} + +function mergeUsage(state: AnthropicUsageState, raw: unknown): void { + if (raw === undefined) return; + const value = object(raw); + if (value === undefined) + throw new AnthropicMessagesCodecError("Anthropic usage must be an object"); + const input = numberField(value.input_tokens, "input_tokens"); + const output = numberField(value.output_tokens, "output_tokens"); + const cacheRead = numberField(value.cache_read_input_tokens, "cache_read_input_tokens"); + const cacheWrite = numberField(value.cache_creation_input_tokens, "cache_creation_input_tokens"); + const details = object(value.cache_creation); + const outputDetails = object(value.output_tokens_details); + const cacheWrite1h = numberField(details?.ephemeral_1h_input_tokens, "ephemeral_1h_input_tokens"); + const reasoning = numberField(outputDetails?.thinking_tokens, "thinking_tokens"); + if (input !== undefined) state.input = input; + if (output !== undefined) state.output = output; + if (cacheRead !== undefined) state.cacheRead = cacheRead; + if (cacheWrite !== undefined) state.cacheWrite = cacheWrite; + if (cacheWrite1h !== undefined) state.cacheWrite1h = cacheWrite1h; + if (reasoning !== undefined) state.reasoning = reasoning; +} + +function usage(state: AnthropicUsageState, model: ModelInfo): Usage { + const mapped: Usage = { + inputTokens: state.input, + outputTokens: state.output, + cacheReadTokens: state.cacheRead, + cacheWriteTokens: state.cacheWrite, + reasoningTokens: state.reasoning, + }; + if (model.cost === undefined) return mapped; + if (state.cacheWrite1h <= 0) return withUsageCost(model.cost, mapped); + if (state.cacheWrite1h > state.cacheWrite) { + throw new AnthropicMessagesCodecError( + "Anthropic 1h cache write tokens exceed total cache write tokens", + ); + } + const rates = modelCostRates(model.cost, mapped); + const shortWrite = state.cacheWrite - state.cacheWrite1h; + return { + ...mapped, + costUsd: + (rates.inputUsdPerMTok * state.input + + rates.outputUsdPerMTok * state.output + + (rates.cacheReadUsdPerMTok ?? 0) * state.cacheRead + + (rates.cacheWriteUsdPerMTok ?? 0) * shortWrite + + rates.inputUsdPerMTok * 2 * state.cacheWrite1h) / + 1_000_000, + }; +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function retryableProviderCode(code: string): boolean { + return new Set(["api_error", "overloaded_error", "rate_limit_error", "timeout_error"]).has(code); +} + +function parseFrame(frame: SseFrame): Record | undefined { + if (frame.event === "ping") return undefined; + try { + const parsed = JSON.parse(frame.data) as unknown; + const value = object(parsed); + if (value === undefined) throw new Error("frame is not an object"); + return value; + } catch (error) { + throw new AnthropicMessagesCodecError("Provider sent an undecodable Anthropic stream frame", { + cause: error, + }); + } +} + +export interface AnthropicMessagesDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +/** Decodes Anthropic Messages SSE frames into canonical stream events. */ +export async function* decodeAnthropicMessagesStream( + frames: AsyncIterable, + options: AnthropicMessagesDecodeOptions, +): AsyncGenerator { + preparedRequest(options.request); + compatibility(options.model); + const blocks = new Map(); + const usageState: AnthropicUsageState = { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + cacheWrite1h: 0, + reasoning: 0, + }; + let responseId: string | undefined; + let routedModelId: string | undefined; + let stopReason: string | undefined; + let stopDetails: Record | undefined; + let emittedContent = false; + let nextContentIndex = 0; + + const responseMetadata = () => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined ? {} : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + ...(stopReason === undefined ? {} : { nativeStopReason: stopReason }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }); + + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if ( + frame.event !== undefined && + !new Set([ + "message_start", + "message_delta", + "message_stop", + "content_block_start", + "content_block_delta", + "content_block_stop", + "error", + ]).has(frame.event) + ) { + continue; + } + const event = parseFrame(frame); + if (event === undefined) continue; + const type = typeof event.type === "string" ? event.type : frame.event; + + if (type === "error") { + const providerError = object(event.error) ?? event; + const code = String(providerError.type ?? providerError.code ?? "provider_error"); + const message = safeProviderMessage( + typeof providerError.message === "string" + ? providerError.message + : "Anthropic reported a failure", + options.secretValues, + ); + yield { + type: "error", + code, + message, + retryable: retryableProviderCode(code), + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + + if (type === "message_start") { + const message = object(event.message); + if (message === undefined) { + throw new AnthropicMessagesCodecError("Anthropic message_start has no message object"); + } + if (typeof message.id === "string" && message.id.length > 0) responseId = message.id; + if (typeof message.model === "string" && message.model.length > 0) + routedModelId = message.model; + mergeUsage(usageState, message.usage); + continue; + } + + if (type === "content_block_start") { + if (!Number.isSafeInteger(event.index) || (event.index as number) < 0) { + throw new AnthropicMessagesCodecError("Anthropic content block has no valid index"); + } + const providerIndex = event.index as number; + if (blocks.has(providerIndex)) { + throw new AnthropicMessagesCodecError( + `Anthropic content block ${providerIndex} started twice`, + ); + } + const content = object(event.content_block); + if (content === undefined || typeof content.type !== "string") { + throw new AnthropicMessagesCodecError("Anthropic content block start is malformed"); + } + const contentIndex = nextContentIndex++; + if (content.type === "text") { + blocks.set(providerIndex, { type: "text", providerIndex, contentIndex }); + if (typeof content.text === "string" && content.text.length > 0) { + emittedContent = true; + yield { type: "text_delta", text: content.text, contentIndex }; + } + } else if (content.type === "thinking") { + const signature = typeof content.signature === "string" ? content.signature : ""; + blocks.set(providerIndex, { + type: "thinking", + providerIndex, + contentIndex, + redacted: false, + signature, + }); + if (typeof content.thinking === "string" && content.thinking.length > 0) { + emittedContent = true; + yield { type: "thinking_delta", text: content.thinking, contentIndex }; + } + } else if (content.type === "redacted_thinking") { + if (typeof content.data !== "string" || content.data.length === 0) { + throw new AnthropicMessagesCodecError("Anthropic redacted thinking has no signature"); + } + blocks.set(providerIndex, { + type: "thinking", + providerIndex, + contentIndex, + redacted: true, + signature: content.data, + }); + emittedContent = true; + yield { type: "thinking_delta", text: "[Reasoning redacted]", contentIndex }; + } else if (content.type === "tool_use") { + if ( + typeof content.id !== "string" || + content.id.length === 0 || + typeof content.name !== "string" || + content.name.length === 0 + ) { + throw new AnthropicMessagesCodecError("Anthropic tool use start is malformed"); + } + const input = object(content.input); + if (content.input !== undefined && input === undefined) { + throw new AnthropicMessagesCodecError("Anthropic tool use input must be an object"); + } + const argumentsText = + input === undefined || Object.keys(input).length === 0 ? "" : JSON.stringify(input); + blocks.set(providerIndex, { + type: "tool", + providerIndex, + contentIndex, + callId: content.id, + wireName: content.name, + argumentsText, + }); + emittedContent = true; + yield { + type: "tool_call_start", + contentIndex, + callId: content.id, + name: reverseToolName(options.request, content.name), + }; + } + continue; + } + + if (type === "content_block_delta") { + if (!Number.isSafeInteger(event.index) || (event.index as number) < 0) { + throw new AnthropicMessagesCodecError("Anthropic content delta has no valid index"); + } + const block = blocks.get(event.index as number); + if (block === undefined) { + throw new AnthropicMessagesCodecError( + `Anthropic delta targets unknown block ${String(event.index)}`, + ); + } + const delta = object(event.delta); + if (delta === undefined || typeof delta.type !== "string") { + throw new AnthropicMessagesCodecError("Anthropic content delta is malformed"); + } + if (delta.type === "text_delta") { + if (block.type !== "text" || typeof delta.text !== "string") { + throw new AnthropicMessagesCodecError("Anthropic text delta targets a non-text block"); + } + if (delta.text.length > 0) { + emittedContent = true; + yield { type: "text_delta", text: delta.text, contentIndex: block.contentIndex }; + } + } else if (delta.type === "thinking_delta") { + if (block.type !== "thinking" || typeof delta.thinking !== "string") { + throw new AnthropicMessagesCodecError( + "Anthropic thinking delta targets a non-thinking block", + ); + } + if (delta.thinking.length > 0) { + emittedContent = true; + yield { + type: "thinking_delta", + text: delta.thinking, + contentIndex: block.contentIndex, + }; + } + } else if (delta.type === "signature_delta") { + if (block.type !== "thinking" || typeof delta.signature !== "string") { + throw new AnthropicMessagesCodecError( + "Anthropic signature delta targets a non-thinking block", + ); + } + block.signature += delta.signature; + } else if (delta.type === "input_json_delta") { + if (block.type !== "tool" || typeof delta.partial_json !== "string") { + throw new AnthropicMessagesCodecError( + "Anthropic tool input delta targets a non-tool block", + ); + } + block.argumentsText += delta.partial_json; + if (delta.partial_json.length > 0) { + yield { + type: "tool_call_delta", + contentIndex: block.contentIndex, + callId: block.callId, + argumentsDelta: delta.partial_json, + }; + } + } + continue; + } + + if (type === "content_block_stop") { + if (!Number.isSafeInteger(event.index) || (event.index as number) < 0) { + throw new AnthropicMessagesCodecError("Anthropic content stop has no valid index"); + } + const block = blocks.get(event.index as number); + if (block === undefined) { + throw new AnthropicMessagesCodecError( + `Anthropic stopped unknown block ${String(event.index)}`, + ); + } + blocks.delete(event.index as number); + if (block.type === "thinking" && block.signature.length > 0) { + yield { + type: "replay_metadata", + target: "thinking", + contentIndex: block.contentIndex, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.request.modelId, + signature: block.signature, + ...(block.redacted ? { redacted: true } : {}), + }; + } else if (block.type === "tool") { + let input: JsonObject; + try { + const parsed = + block.argumentsText.length === 0 ? {} : (JSON.parse(block.argumentsText) as unknown); + const parsedObject = object(parsed); + if (parsedObject === undefined) throw new Error("tool input is not an object"); + input = parsedObject as JsonObject; + } catch (error) { + throw new AnthropicMessagesCodecError( + `Tool call ${block.callId} has undecodable arguments`, + { cause: error }, + ); + } + yield { + type: "tool_call", + contentIndex: block.contentIndex, + callId: block.callId, + name: reverseToolName(options.request, block.wireName), + input, + }; + } + continue; + } + + if (type === "message_delta") { + const delta = object(event.delta); + if (delta === undefined) { + throw new AnthropicMessagesCodecError("Anthropic message_delta has no delta object"); + } + if (delta.stop_reason !== undefined && delta.stop_reason !== null) { + if (typeof delta.stop_reason !== "string" || delta.stop_reason.length === 0) { + throw new AnthropicMessagesCodecError("Anthropic stop reason is malformed"); + } + stopReason = delta.stop_reason; + } + if (delta.stop_details !== undefined && delta.stop_details !== null) { + stopDetails = object(delta.stop_details); + if (stopDetails === undefined) { + throw new AnthropicMessagesCodecError("Anthropic stop details are malformed"); + } + } + mergeUsage(usageState, event.usage); + continue; + } + + if (type === "message_stop") { + if (blocks.size > 0) { + throw new AnthropicMessagesCodecError("Anthropic stopped with incomplete content blocks"); + } + if (stopReason === undefined) { + throw new AnthropicMessagesCodecError("Anthropic stream ended without a stop reason"); + } + const finalUsage = usage(usageState, options.model); + if ( + stopReason === "end_turn" || + stopReason === "stop_sequence" || + stopReason === "pause_turn" + ) { + yield { + type: "completed", + stopReason: "stop", + usage: finalUsage, + response: responseMetadata(), + }; + return; + } + if (stopReason === "max_tokens") { + yield { + type: "completed", + stopReason: "length", + usage: finalUsage, + partial: true, + response: responseMetadata(), + }; + return; + } + if (stopReason === "tool_use") { + yield { + type: "completed", + stopReason: "tool_use", + usage: finalUsage, + response: responseMetadata(), + }; + return; + } + if (stopReason === "refusal" || stopReason === "sensitive") { + const explanation = + typeof stopDetails?.explanation === "string" + ? stopDetails.explanation + : `Provider stopped with: ${stopReason}`; + yield { + type: "error", + code: stopReason, + message: safeProviderMessage(explanation, options.secretValues), + retryable: false, + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + throw new AnthropicMessagesCodecError(`Unhandled Anthropic stop reason: ${stopReason}`); + } + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + } +} diff --git a/packages/ai/src/api-key-auth.ts b/packages/ai/src/api-key-auth.ts new file mode 100644 index 0000000..1629d2c --- /dev/null +++ b/packages/ai/src/api-key-auth.ts @@ -0,0 +1,51 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { ApiKeyAuthMethod } from "./auth.ts"; + +/** Standard stored API key with ordered environment fallback and interactive entry. */ +export function createEnvironmentApiKeyAuth(input: { + readonly providerId: string; + readonly displayName: string; + readonly environmentVariables: readonly string[]; +}): ApiKeyAuthMethod { + return { + displayName: input.displayName, + login: async (interaction) => { + interaction.signal.throwIfAborted(); + const key = await interaction.prompt({ + type: "secret", + message: `Enter ${input.displayName}`, + }); + interaction.signal.throwIfAborted(); + if (key.length === 0) { + throw new TypeError(`${input.providerId} ${input.displayName} cannot be empty`); + } + return { type: "api_key", key }; + }, + resolve: async ({ context, credential, signal }) => { + signal.throwIfAborted(); + if (credential !== undefined) { + if (credential.key === undefined) return undefined; + return { + auth: { apiKey: credential.key }, + source: "stored credential", + ...(credential.env === undefined ? {} : { env: credential.env }), + secretValues: [credential.key], + }; + } + for (const name of input.environmentVariables) { + const key = context.env(name); + signal.throwIfAborted(); + if (key !== undefined && key.length > 0) { + return { + auth: { apiKey: key }, + source: name, + secretValues: [key], + }; + } + } + return undefined; + }, + }; +} diff --git a/packages/ai/src/auth.ts b/packages/ai/src/auth.ts index 1db6341..ba17d60 100644 --- a/packages/ai/src/auth.ts +++ b/packages/ai/src/auth.ts @@ -1,4 +1,5 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 import { access } from "node:fs/promises"; @@ -12,7 +13,9 @@ import { credentialSecretValues, type OAuthCredential, type ProviderEnv, + validateCredential, } from "./credentials.ts"; +import type { AuthMethod } from "./model.ts"; export type AuthErrorCode = "not_configured" | "invalid_auth" | "refresh_failed" | "store_failure"; @@ -29,27 +32,35 @@ export class AuthError extends Error { } /** Request auth for one model call. Anything else is provider configuration. */ +export interface AuthenticatedHttpRequest { + readonly method: string; + readonly url: string; + readonly headers: Readonly>; + readonly body: string; +} + export interface ModelAuth { readonly apiKey?: string; readonly headers?: Readonly>; readonly baseUrl?: string; + /** Provider-owned signer that retains cloud credentials inside its closure. */ + readonly signRequest?: ( + request: AuthenticatedHttpRequest, + signal: AbortSignal, + ) => Promise>>; } export interface ResolvedAuth { readonly auth: ModelAuth; - /** Human-readable origin for status display, e.g. `AZURE_OPENAI_API_KEY`. */ + /** Human-readable, non-secret origin for status display. */ readonly source: string; /** Provider-scoped config resolved alongside the auth. */ readonly env?: ProviderEnv; - /** - * Every secret inside `auth`. Callers must register these with the event - * log's redaction before the auth is used, so credentials can never enter - * prompts, events, generated artifacts, or diagnostics. - */ + /** Every secret inside `auth`, for redaction registration only. */ readonly secretValues: readonly string[]; } -/** Environment access for resolution. Injectable for tests. */ +/** Environment and file availability access for provider-owned resolution. */ export interface AuthContext { env(name: string): string | undefined; fileExists(path: string): Promise; @@ -68,14 +79,61 @@ export const nodeAuthContext: AuthContext = { }, }; -/** - * Api-key-shaped auth: stored keys, environment variables, ambient credential - * files, and keyless local endpoints. `resolve` merges the stored credential - * with ambient sources and returns undefined when the provider is simply not - * configured. - */ +export type AuthPrompt = + | { readonly type: "text"; readonly message: string; readonly placeholder?: string } + | { readonly type: "secret"; readonly message: string; readonly placeholder?: string } + | { + readonly type: "select"; + readonly message: string; + readonly options: readonly { + readonly id: string; + readonly label: string; + readonly description?: string; + }[]; + } + | { + readonly type: "manual_code"; + readonly message: string; + readonly placeholder?: string; + readonly signal?: AbortSignal; + }; + +export type AuthEvent = + | { readonly type: "info"; readonly message: string; readonly links?: readonly AuthInfoLink[] } + | { readonly type: "auth_url"; readonly url: string; readonly instructions?: string } + | { + readonly type: "device_code"; + readonly userCode: string; + readonly verificationUri: string; + readonly intervalSeconds?: number; + readonly expiresInSeconds?: number; + } + | { readonly type: "progress"; readonly message: string } + | { readonly type: "state"; readonly state: AuthenticationState }; + +export interface AuthInfoLink { + readonly url: string; + readonly label?: string; +} + +/** UI-neutral callbacks supplied by a client to a provider-owned login flow. */ +export interface AuthInteraction { + readonly signal?: AbortSignal; + prompt(prompt: AuthPrompt): Promise; + notify(event: AuthEvent): void; +} + +export type ProviderAuthEvent = Exclude; + +export type ProviderAuthInteraction = Omit & { + readonly signal: AbortSignal; + notify(event: ProviderAuthEvent): void; +}; + +/** Api-key-shaped stored auth. Providers may also reuse this shape for ambient sources. */ export interface ApiKeyAuthMethod { readonly displayName: string; + login?(interaction: ProviderAuthInteraction): Promise; resolve(input: { context: AuthContext; credential?: ApiKeyCredential | undefined; @@ -83,20 +141,25 @@ export interface ApiKeyAuthMethod { }): Promise; } -/** - * OAuth-shaped auth. `refresh` exchanges the refresh token (network call, - * throws on failure); `toAuth` derives request auth from a valid credential - * without side effects. The resolver owns the locked refresh pattern. - */ +/** One non-stored source. Resolution always uses the fixed source precedence. */ +export interface AmbientAuthSource extends ApiKeyAuthMethod { + readonly type: "environment" | "file" | "ambient" | "keyless"; +} + export interface OAuthAuthMethod { readonly displayName: string; + /** OAuth may yield refreshable tokens or a provider-issued permanent API key. */ + login?(interaction: ProviderAuthInteraction): Promise; refresh(credential: OAuthCredential, signal: AbortSignal): Promise; - toAuth(credential: OAuthCredential): ModelAuth; + toAuth(credential: OAuthCredential): ModelAuth | Promise; } export interface ProviderAuthMethods { + /** Resolver for a stored api-key credential. */ readonly apiKey?: ApiKeyAuthMethod; readonly oauth?: OAuthAuthMethod; + /** Explicit ambient resolvers, evaluated by type rather than declaration order. */ + readonly sources?: readonly AmbientAuthSource[]; } export interface ResolveAuthOptions { @@ -105,27 +168,246 @@ export interface ResolveAuthOptions { readonly minValidityMs?: number; } +export type AuthenticationPhase = + | "idle" + | "authorizing" + | "authenticated" + | "reauthentication_required" + | "logged_out"; + +/** Public authentication state contains no credential or provider response values. */ +export interface AuthenticationState { + readonly phase: AuthenticationPhase; + readonly method?: "api_key" | "oauth"; + readonly source?: string; +} + +export interface ProviderAuthentication { + readonly methods: readonly AuthMethod[]; + readonly loginMethods: readonly ("api_key" | "oauth")[]; + state(): AuthenticationState; + /** Checks configured authentication without refreshing stored OAuth credentials. */ + check(options?: { readonly signal?: AbortSignal }): Promise; + resolve(options?: ResolveAuthOptions): Promise; + login(method: "api_key" | "oauth", interaction: AuthInteraction): Promise; + logout(options?: { readonly signal?: AbortSignal }): Promise; +} + const DEFAULT_MIN_VALIDITY_MS = 5 * 60 * 1000; const REFRESH_TIMEOUT_MS = 15_000; +const SOURCE_PRECEDENCE: Readonly> = { + environment: 0, + file: 1, + ambient: 2, + keyless: 3, +}; -/** Explicit login: persist a credential through the store's serialized write path. */ +/** Explicit credential persistence through the store's serialized write path. */ export async function login( store: CredentialStore, providerId: string, credential: Credential, ): Promise { - await store.modify(providerId, () => Promise.resolve(credential)); + const validated = validateCredential(credential, providerId); + await store.modify(providerId, () => Promise.resolve(validated)); } -/** Explicit logout: remove the stored credential. */ +/** Explicit logout, serialized against refresh and login writes. */ export async function logout(store: CredentialStore, providerId: string): Promise { await store.delete(providerId); } /** - * Resolves request auth for a provider or fails with an explicit auth state. - * A stored credential owns the provider; ambient sources are consulted only - * when nothing is stored, and never after a failed refresh. + * Provider-owned authentication lifecycle. A monotonic generation prevents a + * cancelled or superseded login or refresh from publishing authenticated + * state, and prevents login completion from restoring a credential after + * logout. + */ +export function createProviderAuthentication(input: { + readonly providerId: string; + readonly declaredMethods: readonly AuthMethod[]; + readonly methods: ProviderAuthMethods; + readonly store: CredentialStore; + readonly context: AuthContext; +}): ProviderAuthentication { + let current: AuthenticationState = { phase: "idle" }; + let generation = 0; + + const transition = (state: AuthenticationState, interaction?: AuthInteraction) => { + current = state; + interaction?.notify({ type: "state", state }); + return state; + }; + + return { + methods: [...input.declaredMethods], + loginMethods: [ + ...(input.methods.apiKey?.login === undefined ? [] : (["api_key"] as const)), + ...(input.methods.oauth?.login === undefined ? [] : (["oauth"] as const)), + ], + state: () => ({ ...current }), + check: async (options = {}) => { + const signal = options.signal ?? new AbortController().signal; + signal.throwIfAborted(); + let stored: Credential | undefined; + try { + stored = await input.store.read(input.providerId); + } catch (error) { + throw new AuthError( + "store_failure", + input.providerId, + `Credential store read failed for ${input.providerId}`, + error, + ); + } + signal.throwIfAborted(); + if (stored?.type === "oauth" && input.methods.oauth !== undefined) { + return transition({ + phase: "authenticated", + method: "oauth", + source: input.methods.oauth.displayName, + }); + } + if (stored?.type === "api_key" && input.methods.apiKey !== undefined) { + return transition({ + phase: "authenticated", + method: "api_key", + source: input.methods.apiKey.displayName, + }); + } + if (stored !== undefined) { + return transition({ phase: "reauthentication_required", method: stored.type }); + } + try { + const resolved = await resolveProviderAuth( + input.providerId, + input.methods, + input.store, + input.context, + { signal }, + ); + signal.throwIfAborted(); + return transition({ phase: "authenticated", source: resolved.source }); + } catch (error) { + if (error instanceof AuthError && error.code === "not_configured") { + return transition({ phase: "logged_out" }); + } + throw error; + } + }, + resolve: async (options = {}) => { + const operation = generation; + try { + const resolved = await resolveProviderAuth( + input.providerId, + input.methods, + input.store, + input.context, + options, + ); + options.signal?.throwIfAborted(); + if (operation !== generation) { + throw new AuthError( + "not_configured", + input.providerId, + `Authentication changed while resolving ${input.providerId}`, + ); + } + transition({ phase: "authenticated", source: resolved.source }); + return resolved; + } catch (error) { + if (error instanceof AuthError && error.code === "refresh_failed") { + transition({ phase: "reauthentication_required", method: "oauth" }); + } + throw error; + } + }, + login: async (method, interaction) => { + const previous = current; + const operation = ++generation; + const signal = interaction.signal ?? new AbortController().signal; + signal.throwIfAborted(); + transition({ phase: "authorizing", method }, interaction); + const implementation = method === "oauth" ? input.methods.oauth : input.methods.apiKey; + if (implementation?.login === undefined) { + transition({ phase: "reauthentication_required", method }, interaction); + throw new AuthError( + "not_configured", + input.providerId, + `${implementation?.displayName ?? method} does not support interactive login`, + ); + } + try { + const credential = validateCredential( + await implementation.login({ + signal, + prompt: async (prompt) => { + signal.throwIfAborted(); + const answer = await interaction.prompt(prompt); + signal.throwIfAborted(); + if (typeof answer !== "string") { + throw new AuthError( + "invalid_auth", + input.providerId, + `Authentication prompt returned a malformed response for ${input.providerId}`, + ); + } + return answer; + }, + notify: (event) => { + signal.throwIfAborted(); + interaction.notify(validateAuthEvent(event, input.providerId)); + }, + }), + input.providerId, + ); + signal.throwIfAborted(); + await input.store.modify(input.providerId, () => { + signal.throwIfAborted(); + return Promise.resolve(operation === generation ? credential : undefined); + }); + signal.throwIfAborted(); + if (operation !== generation) { + throw new AuthError( + "not_configured", + input.providerId, + `Authentication was superseded for ${input.providerId}`, + ); + } + return transition( + { phase: "authenticated", method, source: implementation.displayName }, + interaction, + ); + } catch (error) { + if (operation === generation) { + transition( + signal.aborted ? previous : { phase: "reauthentication_required", method }, + interaction, + ); + } + if (signal.aborted) signal.throwIfAborted(); + if (error instanceof AuthError) throw error; + throw new AuthError( + "invalid_auth", + input.providerId, + `${implementation.displayName} login failed for ${input.providerId}`, + ); + } + }, + logout: async (options = {}) => { + generation += 1; + options.signal?.throwIfAborted(); + await input.store.delete(input.providerId); + options.signal?.throwIfAborted(); + return transition({ phase: "logged_out" }); + }, + }; +} + +/** + * Resolves request auth or fails with an explicit state. Stored credentials + * own the provider. Without one, ambient sources use environment, file, + * ambient, then keyless precedence regardless of declaration order. */ export async function resolveProviderAuth( providerId: string, @@ -163,40 +445,61 @@ export async function resolveProviderAuth( ); } - if (methods.apiKey) { - return resolveApiKey(providerId, methods.apiKey, context, undefined, signal); + if (methods.sources !== undefined) { + const sources = [...methods.sources].sort( + (left, right) => SOURCE_PRECEDENCE[left.type] - SOURCE_PRECEDENCE[right.type], + ); + for (const source of sources) { + const resolved = await tryResolveApiKey(providerId, source, context, undefined, signal); + if (resolved !== undefined) return resolved; + } + } else if (methods.apiKey) { + const resolved = await tryResolveApiKey(providerId, methods.apiKey, context, undefined, signal); + if (resolved !== undefined) return resolved; } + throw new AuthError( "not_configured", providerId, - `Provider ${providerId} has no stored credential and no ambient auth method`, + `Provider ${providerId} is not configured with a stored credential or ambient auth method`, ); } -async function resolveApiKey( +async function tryResolveApiKey( providerId: string, method: ApiKeyAuthMethod, context: AuthContext, credential: ApiKeyCredential | undefined, signal: AbortSignal, -): Promise { - let resolved: ResolvedAuth | undefined; +): Promise { try { - resolved = await method.resolve({ context, credential, signal }); + const resolved = await method.resolve({ context, credential, signal }); + signal.throwIfAborted(); + return resolved === undefined ? undefined : validateResolvedAuth(resolved, providerId); } catch (error) { + if (signal.aborted) signal.throwIfAborted(); if (error instanceof AuthError) throw error; throw new AuthError( "invalid_auth", providerId, `${method.displayName} resolution failed for ${providerId}`, - error, ); } +} + +async function resolveApiKey( + providerId: string, + method: ApiKeyAuthMethod, + context: AuthContext, + credential: ApiKeyCredential | undefined, + signal: AbortSignal, +): Promise { + const resolved = await tryResolveApiKey(providerId, method, context, credential, signal); if (resolved === undefined) { throw new AuthError( - "not_configured", + "invalid_auth", providerId, - `${method.displayName} is not configured for ${providerId}`, + `${method.displayName} rejected the stored credential for ${providerId}`, ); } return resolved; @@ -216,22 +519,22 @@ async function resolveStoredOAuth( let credential = stored; if (expiresSoon(credential)) { - // The optimistic check saw an expiring token; the authoritative check and - // the single refresh both run inside the store's serialized write. let post: Credential | undefined; try { post = await store.modify(providerId, async (current) => { - if (current?.type !== "oauth") return undefined; // logged out meanwhile - if (!expiresSoon(current)) return undefined; // already refreshed elsewhere + if (current?.type !== "oauth") return undefined; + if (!expiresSoon(current)) return undefined; const refreshSignal = AbortSignal.any([signal, AbortSignal.timeout(REFRESH_TIMEOUT_MS)]); - return method.refresh(current, refreshSignal); + const refreshed = await method.refresh(current, refreshSignal); + refreshSignal.throwIfAborted(); + return validateCredential(refreshed, providerId); }); - } catch (error) { + } catch { + if (signal.aborted) signal.throwIfAborted(); throw new AuthError( "refresh_failed", providerId, `OAuth refresh failed for ${providerId}; log in again`, - error, ); } if (post?.type !== "oauth") { @@ -244,9 +547,118 @@ async function resolveStoredOAuth( credential = post; } - return { - auth: method.toAuth(credential), - source: method.displayName, - secretValues: credentialSecretValues(credential), - }; + try { + const auth = await method.toAuth(credential); + const secretValues = [...credentialSecretValues(credential)]; + for (const value of [auth.apiKey, ...Object.values(auth.headers ?? {})]) { + if ( + value !== undefined && + value.length > 0 && + !secretValues.some((secret) => value.includes(secret)) + ) { + secretValues.push(value); + } + } + return validateResolvedAuth( + { + auth, + source: method.displayName, + secretValues, + }, + providerId, + ); + } catch (error) { + if (error instanceof AuthError) throw error; + throw new AuthError( + "invalid_auth", + providerId, + `OAuth request authentication failed for ${providerId}; log in again`, + ); + } +} + +function validateResolvedAuth(value: ResolvedAuth, providerId: string): ResolvedAuth { + if (!isPlainObject(value) || !isPlainObject(value.auth)) { + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} returned malformed auth`, + ); + } + const allowedResult = new Set(["auth", "source", "env", "secretValues"]); + const allowedAuth = new Set(["apiKey", "headers", "baseUrl", "signRequest"]); + if ( + Object.keys(value).some((key) => !allowedResult.has(key)) || + Object.keys(value.auth).some((key) => !allowedAuth.has(key)) || + typeof value.source !== "string" || + value.source.length === 0 || + !Array.isArray(value.secretValues) || + value.secretValues.some((secret) => typeof secret !== "string" || secret.length === 0) || + value.secretValues.some((secret) => value.source.includes(secret)) || + (value.auth.apiKey !== undefined && typeof value.auth.apiKey !== "string") || + (value.auth.baseUrl !== undefined && typeof value.auth.baseUrl !== "string") || + (value.auth.signRequest !== undefined && typeof value.auth.signRequest !== "function") || + !isStringRecord(value.auth.headers) || + !isStringRecord(value.env) + ) { + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} returned malformed auth`, + ); + } + const protectedValues = [value.auth.apiKey, ...Object.values(value.auth.headers ?? {})].filter( + (item): item is string => item !== undefined && item.length > 0, + ); + if ( + protectedValues.some( + (protectedValue) => !value.secretValues.some((secret) => protectedValue.includes(secret)), + ) + ) { + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} omitted authentication values from redaction`, + ); + } + return value; +} + +function validateAuthEvent(event: ProviderAuthEvent, providerId: string): ProviderAuthEvent { + if (!isPlainObject(event) || typeof event.type !== "string") { + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} emitted a malformed authentication event`, + ); + } + if (event.type === "info" || event.type === "progress") { + if (typeof event.message === "string") return event; + } else if (event.type === "auth_url") { + if (typeof event.url === "string") return event; + } else if (event.type === "device_code") { + if (typeof event.userCode === "string" && typeof event.verificationUri === "string") + return event; + } + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} emitted a malformed authentication event`, + ); +} + +function isPlainObject(value: unknown): value is Record { + return ( + typeof value === "object" && + value !== null && + !Array.isArray(value) && + Object.getPrototypeOf(value) === Object.prototype + ); +} + +function isStringRecord(value: unknown): value is Readonly> | undefined { + return ( + value === undefined || + (isPlainObject(value) && Object.values(value).every((item) => typeof item === "string")) + ); } diff --git a/packages/ai/src/aws-auth.ts b/packages/ai/src/aws-auth.ts new file mode 100644 index 0000000..3e0b830 --- /dev/null +++ b/packages/ai/src/aws-auth.ts @@ -0,0 +1,226 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { defaultProvider } from "@aws-sdk/credential-provider-node"; +import { Hash } from "@smithy/hash-node"; +import { HttpRequest } from "@smithy/protocol-http"; +import { SignatureV4 } from "@smithy/signature-v4"; + +import { + type AmbientAuthSource, + type ApiKeyAuthMethod, + type AuthenticatedHttpRequest, + type AuthContext, + AuthError, + type ResolvedAuth, +} from "./auth.ts"; +import type { ApiKeyCredential } from "./credentials.ts"; +import { raceWithSignal } from "./transport-safety.ts"; + +export interface AwsCredentialIdentityLike { + readonly accessKeyId: string; + readonly secretAccessKey: string; + readonly sessionToken?: string; + readonly expiration?: Date; +} + +export type AwsCredentialProviderLike = () => Promise; + +export interface AwsAuthFactories { + readonly credentials?: (options: { readonly profile?: string }) => AwsCredentialProviderLike; +} + +function region(context: AuthContext, credential?: ApiKeyCredential): string { + const value = + credential?.env?.AWS_REGION ?? + credential?.env?.AWS_DEFAULT_REGION ?? + context.env("AWS_REGION") ?? + context.env("AWS_DEFAULT_REGION"); + if (!value?.trim()) { + throw new AuthError( + "not_configured", + "amazon-bedrock", + "Amazon Bedrock requires an AWS region", + ); + } + return value.trim(); +} + +function profile(context: AuthContext, credential?: ApiKeyCredential): string | undefined { + return credential?.env?.AWS_PROFILE?.trim() || context.env("AWS_PROFILE")?.trim() || undefined; +} + +function query(url: URL): Record { + const result: Record = {}; + for (const [name, value] of url.searchParams) { + const current = result[name]; + if (current === undefined) result[name] = value; + else if (Array.isArray(current)) current.push(value); + else result[name] = [current, value]; + } + return result; +} + +function validateCredentials(value: AwsCredentialIdentityLike): AwsCredentialIdentityLike { + if (!value.accessKeyId || !value.secretAccessKey) { + throw new AuthError( + "invalid_auth", + "amazon-bedrock", + "AWS credential chain returned incomplete credentials", + ); + } + return value; +} + +async function sign( + request: AuthenticatedHttpRequest, + signal: AbortSignal, + credentials: AwsCredentialIdentityLike, + awsRegion: string, +): Promise>> { + signal.throwIfAborted(); + const url = new URL(request.url); + const signer = new SignatureV4({ + credentials, + region: awsRegion, + service: "bedrock", + sha256: Hash.bind(null, "sha256"), + }); + const signed = await signer.sign( + new HttpRequest({ + protocol: url.protocol, + hostname: url.hostname, + ...(url.port ? { port: Number(url.port) } : {}), + method: request.method, + path: url.pathname, + query: query(url), + headers: { host: url.host, ...request.headers }, + body: request.body, + }), + ); + signal.throwIfAborted(); + return signed.headers; +} + +function bearer(context: AuthContext, credential?: ApiKeyCredential): ResolvedAuth | undefined { + const token = credential?.key ?? context.env("AWS_BEARER_TOKEN_BEDROCK"); + if (!token) return undefined; + return { + auth: { apiKey: token }, + env: { AWS_REGION: region(context, credential) }, + source: credential?.key === undefined ? "AWS_BEARER_TOKEN_BEDROCK" : "stored credential", + secretValues: [token], + }; +} + +async function sigv4( + context: AuthContext, + credential: ApiKeyCredential | undefined, + signal: AbortSignal, + factories: AwsAuthFactories, +): Promise { + const awsRegion = region(context, credential); + const selectedProfile = profile(context, credential); + const provider = + factories.credentials?.(selectedProfile === undefined ? {} : { profile: selectedProfile }) ?? + defaultProvider(selectedProfile === undefined ? {} : { profile: selectedProfile }); + let credentials: AwsCredentialIdentityLike; + try { + signal.throwIfAborted(); + credentials = validateCredentials(await raceWithSignal(provider(), signal)); + signal.throwIfAborted(); + } catch (cause) { + if (signal.aborted) signal.throwIfAborted(); + if (cause instanceof AuthError) throw cause; + throw new AuthError( + "invalid_auth", + "amazon-bedrock", + "AWS default credential chain failed for amazon-bedrock", + cause, + ); + } + const secretValues = [ + credentials.accessKeyId, + credentials.secretAccessKey, + ...(credentials.sessionToken === undefined ? [] : [credentials.sessionToken]), + ]; + return { + auth: { + signRequest: (request, requestSignal) => sign(request, requestSignal, credentials, awsRegion), + }, + env: { AWS_REGION: awsRegion, ...(selectedProfile ? { AWS_PROFILE: selectedProfile } : {}) }, + source: selectedProfile ? `AWS profile ${selectedProfile}` : "AWS default credential chain", + secretValues, + }; +} + +export function createBedrockStoredAuth(factories: AwsAuthFactories = {}): ApiKeyAuthMethod { + return { + displayName: "Amazon Bedrock credentials", + login: async (interaction) => { + const method = await interaction.prompt({ + type: "select", + message: "Select Amazon Bedrock authentication method:", + options: [ + { id: "bearer", label: "Bedrock bearer token" }, + { id: "profile", label: "AWS profile" }, + { id: "default", label: "AWS default credential chain" }, + ], + }); + const awsRegion = await interaction.prompt({ + type: "text", + message: "Enter AWS region:", + }); + if (method === "bearer") { + return { + type: "api_key", + key: await interaction.prompt({ + type: "secret", + message: "Enter Amazon Bedrock bearer token:", + }), + env: { AWS_REGION: awsRegion }, + }; + } + if (method === "profile") { + return { + type: "api_key", + env: { + AWS_REGION: awsRegion, + AWS_PROFILE: await interaction.prompt({ + type: "text", + message: "Enter AWS profile name:", + }), + }, + }; + } + if (method !== "default") throw new Error(`Unknown Amazon Bedrock auth method: ${method}`); + return { type: "api_key", env: { AWS_REGION: awsRegion } }; + }, + resolve: async ({ context, credential, signal }) => { + const token = bearer(context, credential); + if (token !== undefined) return token; + if (credential === undefined) return undefined; + return sigv4(context, credential, signal, factories); + }, + }; +} + +export function createBedrockSources( + factories: AwsAuthFactories = {}, +): readonly AmbientAuthSource[] { + return [ + { + type: "environment", + displayName: "Amazon Bedrock bearer token", + resolve: async ({ context, signal }) => { + signal.throwIfAborted(); + return bearer(context); + }, + }, + { + type: "ambient", + displayName: "AWS default credential chain", + resolve: ({ context, signal }) => sigv4(context, undefined, signal, factories), + }, + ]; +} diff --git a/packages/ai/src/aws-event-stream.ts b/packages/ai/src/aws-event-stream.ts new file mode 100644 index 0000000..e6134e1 --- /dev/null +++ b/packages/ai/src/aws-event-stream.ts @@ -0,0 +1,89 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +const decoder = new TextDecoder(); +export const MAX_AWS_EVENT_STREAM_FRAME_BYTES = 16 * 1024 * 1024; +export const MAX_AWS_EVENT_STREAM_TOTAL_BYTES = 64 * 1024 * 1024; + +function crc32(bytes: Uint8Array): number { + let crc = 0xffffffff; + for (const byte of bytes) { + crc ^= byte; + for (let bit = 0; bit < 8; bit += 1) crc = (crc >>> 1) ^ (0xedb88320 & -(crc & 1)); + } + return (crc ^ 0xffffffff) >>> 0; +} + +function eventType(headers: Uint8Array): string { + let offset = 0; + while (offset < headers.length) { + const nameLength = headers[offset]; + if (nameLength === undefined || offset + 2 + nameLength > headers.length) + throw new TypeError("AWS event stream header is truncated"); + const name = decoder.decode(headers.subarray(offset + 1, offset + 1 + nameLength)); + const type = headers[offset + 1 + nameLength]; + offset += 2 + nameLength; + if (type !== 7) throw new TypeError("AWS event stream uses an unsupported header type"); + if (offset + 2 > headers.length) + throw new TypeError("AWS event stream header value is truncated"); + const length = new DataView(headers.buffer, headers.byteOffset + offset, 2).getUint16(0); + offset += 2; + if (offset + length > headers.length) + throw new TypeError("AWS event stream header value is truncated"); + const value = decoder.decode(headers.subarray(offset, offset + length)); + offset += length; + if (name === ":event-type" || name === ":exception-type") return value; + } + throw new TypeError("AWS event stream message has no event type"); +} + +/** Decodes checked AWS event-stream messages into SDK-shaped Bedrock events. */ +export async function* decodeAwsEventStream( + body: ReadableStream, +): AsyncGenerator> { + let storage = new Uint8Array(64 * 1024); + let start = 0; + let end = 0; + let received = 0; + for await (const chunk of body) { + received += chunk.byteLength; + if (received > MAX_AWS_EVENT_STREAM_TOTAL_BYTES) + throw new TypeError("AWS event stream response exceeds its byte limit"); + const pending = end - start; + if (pending + chunk.byteLength > MAX_AWS_EVENT_STREAM_FRAME_BYTES) + throw new TypeError("AWS event stream pending data exceeds its byte limit"); + if (storage.length - end < chunk.byteLength) { + const capacity = Math.max(storage.length * 2, pending + chunk.byteLength); + const next = new Uint8Array(capacity); + next.set(storage.subarray(start, end)); + storage = next; + end = pending; + start = 0; + } + storage.set(chunk, end); + end += chunk.byteLength; + while (end - start >= 16) { + const view = new DataView(storage.buffer, storage.byteOffset + start, end - start); + const total = view.getUint32(0); + const headersLength = view.getUint32(4); + if (total < 16 || total > MAX_AWS_EVENT_STREAM_FRAME_BYTES || headersLength > total - 16) + throw new TypeError("AWS event stream message has invalid or oversized lengths"); + if (end - start < total) break; + const message = storage.subarray(start, start + total); + start += total; + const messageView = new DataView(message.buffer, message.byteOffset, message.byteLength); + if (crc32(message.subarray(0, 8)) !== messageView.getUint32(8)) + throw new TypeError("AWS event stream prelude checksum failed"); + if (crc32(message.subarray(0, total - 4)) !== messageView.getUint32(total - 4)) + throw new TypeError("AWS event stream message checksum failed"); + const type = eventType(message.subarray(12, 12 + headersLength)); + const payloadText = decoder.decode(message.subarray(12 + headersLength, total - 4)); + const payload = payloadText.length === 0 ? {} : (JSON.parse(payloadText) as unknown); + if (typeof payload !== "object" || payload === null || Array.isArray(payload)) + throw new TypeError("AWS event stream payload is malformed"); + yield { [type]: payload }; + if (start === end) start = end = 0; + } + } + if (end !== start) throw new TypeError("AWS event stream ended with a partial message"); +} diff --git a/packages/ai/src/azure-openai-models.ts b/packages/ai/src/azure-openai-models.ts index e8ee581..ba954e1 100644 --- a/packages/ai/src/azure-openai-models.ts +++ b/packages/ai/src/azure-openai-models.ts @@ -5,7 +5,7 @@ import type { ModelInfo } from "./model.ts"; const defaults = { providerId: "azure-openai", - apiDialect: "openai-responses", + apiDialect: "azure-openai-responses", capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, } as const; @@ -45,7 +45,14 @@ function azureModel(definition: AzureModelDefinition): ModelInfo { ...(definition.thinkingLevelMap === undefined ? {} : { thinkingLevelMap: definition.thinkingLevelMap }), - ...(definition.grammarTools ? { compatibility: { supportsOpenAIGrammarTools: true } } : {}), + ...(definition.grammarTools + ? { + compatibility: { + dialect: "azure-openai-responses" as const, + supportsGrammarTools: true, + }, + } + : {}), }; } diff --git a/packages/ai/src/azure-openai.ts b/packages/ai/src/azure-openai.ts index 5f9e321..d45aaed 100644 --- a/packages/ai/src/azure-openai.ts +++ b/packages/ai/src/azure-openai.ts @@ -3,25 +3,43 @@ // Axl-native Azure OpenAI endpoint, authentication, and deployment mapping. -import { AuthError, type ApiKeyAuthMethod, type AuthContext, type ResolvedAuth } from "./auth.ts"; +import { + type ApiKeyAuthMethod, + type AuthContext, + AuthError, + createProviderAuthentication, + type ResolvedAuth, +} from "./auth.ts"; import type { CredentialStore } from "./credentials.ts"; -import { resolveProviderAuth } from "./auth.ts"; import type { ModelInfo } from "./model.ts"; -import { OpenAiResponsesProvider, type ResponsesEndpoint } from "./openai-responses.ts"; +import { + type EncodedResponsesRequest, + encodeResponsesRequest, + OpenAiResponsesProvider, + type ResponsesEndpoint, +} from "./openai-responses.ts"; +import { safeFetch } from "./transport-safety.ts"; +import type { PreparedModelRequest } from "./request-preparation.ts"; export const AZURE_OPENAI_PROVIDER_ID = "azure-openai"; import { AZURE_OPENAI_MODELS } from "./azure-openai-models.ts"; export { AZURE_OPENAI_MODELS }; -const DEFAULT_API_VERSION = "v1"; +export const DEFAULT_AZURE_OPENAI_API_VERSION = "v1"; + +function stripTrailingSlashes(value: string): string { + let end = value.length; + while (end > 0 && value.charCodeAt(end - 1) === 47) end -= 1; + return value.slice(0, end); +} /** * Normalizes an Azure OpenAI base URL. Azure hosts get the `/openai/v1` base * path; non-Azure hosts (gateways, proxies) pass through untouched. */ export function normalizeAzureBaseUrl(baseUrl: string): string { - const trimmed = baseUrl.trim().replace(/\/+$/, ""); + const trimmed = stripTrailingSlashes(baseUrl.trim()); let url: URL; try { url = new URL(trimmed); @@ -37,12 +55,12 @@ export function normalizeAzureBaseUrl(baseUrl: string): string { url.hostname.endsWith(".openai.azure.com") || url.hostname.endsWith(".cognitiveservices.azure.com") || url.hostname.endsWith(".ai.azure.com"); - const path = url.pathname.replace(/\/+$/, ""); + const path = stripTrailingSlashes(url.pathname); if (isAzureHost && (path === "" || path === "/openai" || path === "/openai/v1/responses")) { url.pathname = "/openai/v1"; url.search = ""; } - return url.toString().replace(/\/+$/, ""); + return stripTrailingSlashes(url.toString()); } /** Parses the `model=deployment,...` map from AZURE_OPENAI_DEPLOYMENT_NAME_MAP. */ @@ -100,20 +118,34 @@ export const azureOpenAiAuthMethod: ApiKeyAuthMethod = { }, }; +function resolvedAzureBaseUrl(resolved: ResolvedAuth): string { + const base = resolved.auth.baseUrl ?? resolved.env?.AZURE_OPENAI_BASE_URL; + if (base === undefined) { + throw new AuthError( + "not_configured", + AZURE_OPENAI_PROVIDER_ID, + "Azure OpenAI base URL missing from resolved auth", + ); + } + return normalizeAzureBaseUrl(base); +} + +function resolvedAzureApiVersion(resolved: ResolvedAuth): string { + const version = + resolved.env?.AZURE_OPENAI_API_VERSION?.trim() || DEFAULT_AZURE_OPENAI_API_VERSION; + return version; +} + +function azureResourceUrl(resolved: ResolvedAuth, resource: "responses" | "models"): string { + const url = new URL(resolvedAzureBaseUrl(resolved)); + url.pathname = `${stripTrailingSlashes(url.pathname)}/${resource}`; + url.searchParams.set("api-version", resolvedAzureApiVersion(resolved)); + return url.toString(); +} + /** Azure endpoint policy over the resolved auth: URL, api-key header, deployments. */ export const azureEndpoint: ResponsesEndpoint = { - url: (resolved: ResolvedAuth): string => { - const base = resolved.auth.baseUrl ?? resolved.env?.AZURE_OPENAI_BASE_URL; - if (base === undefined) { - throw new AuthError( - "not_configured", - AZURE_OPENAI_PROVIDER_ID, - "Azure OpenAI base URL missing from resolved auth", - ); - } - const version = resolved.env?.AZURE_OPENAI_API_VERSION ?? DEFAULT_API_VERSION; - return `${base}/responses?api-version=${encodeURIComponent(version)}`; - }, + url: (resolved: ResolvedAuth): string => azureResourceUrl(resolved, "responses"), headers: (resolved: ResolvedAuth): Readonly> => ({ ...(resolved.auth.apiKey === undefined ? {} : { "api-key": resolved.auth.apiKey }), ...resolved.auth.headers, @@ -122,6 +154,31 @@ export const azureEndpoint: ResponsesEndpoint = { parseDeploymentMap(resolved.env?.AZURE_OPENAI_DEPLOYMENT_NAME_MAP)[modelId] ?? modelId, }; +export interface EncodedAzureOpenAiResponsesRequest extends EncodedResponsesRequest { + readonly url: string; +} + +/** Composes one prepared request with Azure deployment, version, and header policy. */ +export function encodeAzureOpenAiResponsesRequest( + model: ModelInfo, + request: PreparedModelRequest, + resolved: ResolvedAuth, +): EncodedAzureOpenAiResponsesRequest { + if (model.apiDialect !== "azure-openai-responses") { + throw new TypeError(`Model ${model.modelId} does not use the Azure OpenAI Responses dialect`); + } + const encoded = encodeResponsesRequest( + model, + request, + azureEndpoint.deploymentFor(model.modelId, resolved), + ); + return { + url: azureEndpoint.url(resolved), + headers: { ...encoded.headers, ...azureEndpoint.headers(resolved) }, + body: encoded.body, + }; +} + export interface AzureVerification { readonly ok: boolean; readonly status?: number; @@ -139,11 +196,17 @@ export async function verifyAzureOpenAiAuth( ): Promise { const base = resolved.auth.baseUrl ?? resolved.env?.AZURE_OPENAI_BASE_URL; if (base === undefined) return { ok: false, detail: "no base URL resolved" }; - const version = resolved.env?.AZURE_OPENAI_API_VERSION ?? DEFAULT_API_VERSION; try { - const response = await fetchImpl(`${base}/models?api-version=${encodeURIComponent(version)}`, { - headers: azureEndpoint.headers(resolved), - }); + const url = azureResourceUrl(resolved, "models"); + const response = await safeFetch( + url, + { headers: azureEndpoint.headers(resolved) }, + { + label: "Azure OpenAI verification endpoint", + expectedOrigin: new URL(url).origin, + ...(fetchImpl === fetch ? {} : { fetch: fetchImpl }), + }, + ); if (response.ok) return { ok: true, status: response.status }; const detail = (await response.text().catch(() => "")).slice(0, 300); return { ok: false, status: response.status, ...(detail ? { detail } : {}) }; @@ -163,19 +226,21 @@ export interface AzureOpenAiProviderOptions { export function createAzureOpenAiProvider( options: AzureOpenAiProviderOptions, ): OpenAiResponsesProvider { + const authentication = createProviderAuthentication({ + providerId: AZURE_OPENAI_PROVIDER_ID, + declaredMethods: ["environment", "file"], + methods: { apiKey: azureOpenAiAuthMethod }, + store: options.store, + context: options.context, + }); return new OpenAiResponsesProvider({ id: AZURE_OPENAI_PROVIDER_ID, displayName: "Azure OpenAI", - authMethods: ["environment", "file"], + authMethods: authentication.methods, + authentication, endpoint: azureEndpoint, models: options.models ?? AZURE_OPENAI_MODELS, - resolveAuth: () => - resolveProviderAuth( - AZURE_OPENAI_PROVIDER_ID, - { apiKey: azureOpenAiAuthMethod }, - options.store, - options.context, - ), + resolveAuth: () => authentication.resolve(), ...(options.fetch === undefined ? {} : { fetch: options.fetch }), }); } diff --git a/packages/ai/src/baseten.ts b/packages/ai/src/baseten.ts new file mode 100644 index 0000000..dda2c7a --- /dev/null +++ b/packages/ai/src/baseten.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const BASETEN_PROVIDER_ID = "baseten"; +export const BASETEN_API_KEY_ENV = "BASETEN_API_KEY"; +export const BASETEN_BASE_URL = "https://inference.baseten.co/v1"; + +export const BASETEN_PROVIDER_DEFINITION = { + id: BASETEN_PROVIDER_ID, + displayName: "Baseten", + apiKeyDisplayName: "Baseten API key", + environmentVariables: [BASETEN_API_KEY_ENV], + baseUrl: BASETEN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createBasetenProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(BASETEN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/bedrock-converse-stream.ts b/packages/ai/src/bedrock-converse-stream.ts new file mode 100644 index 0000000..3cbf866 --- /dev/null +++ b/packages/ai/src/bedrock-converse-stream.ts @@ -0,0 +1,926 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Transport-neutral Amazon Bedrock Converse Stream request and event codec. + +import type { JsonObject, JsonValue, Usage } from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { BedrockCompatibility, ModelInfo, ModelStreamEvent } from "./model.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + type PreparedRequestMessage, +} from "./request-preparation.ts"; +import { stripTrailingSlashes } from "./transport-safety.ts"; +import { withUsageCost } from "./usage.ts"; + +const EMPTY_TEXT_PLACEHOLDER = ""; +const REDACTED_THINKING_PLACEHOLDER = "[Reasoning redacted]"; +const REGION_PATTERN = /^[a-z]{2}(?:-gov)?-[a-z]+-\d$/; +const RESERVED_REQUEST_FIELDS = new Set(["thinking", "output_config", "anthropic_beta"]); + +export class BedrockConverseStreamCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "BedrockConverseStreamCodecError"; + } +} + +export type BedrockAuthenticationPolicy = + | { readonly type: "sigv4" } + | { readonly type: "bearer"; readonly token: string }; + +export interface BedrockRequestPolicy { + readonly region?: string; + readonly baseUrl?: string; + readonly authentication: BedrockAuthenticationPolicy; + readonly thinkingDisplay?: "summarized" | "omitted"; + readonly interleavedThinking?: boolean; +} + +export interface AwsSigningInputs { + readonly service: "bedrock"; + readonly region: string; +} + +export interface EncodedBedrockConverseStreamRequest { + readonly method: "POST"; + readonly url: string; + readonly body: JsonObject; + readonly headers: Readonly>; + /** Credentials and the signature itself are supplied by the transport layer. */ + readonly signing?: AwsSigningInputs; +} + +export type BedrockConverseStreamEvent = Readonly>; + +export interface BedrockConverseStreamDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly responseId?: string; + readonly routedModelId?: string; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +type MutableJsonObject = Record; + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function compatibility(model: ModelInfo): BedrockCompatibility { + if (model.apiDialect !== "bedrock-converse-stream") { + throw new BedrockConverseStreamCodecError( + `Model ${model.modelId} does not use the bedrock-converse-stream dialect`, + ); + } + if (model.compatibility?.dialect !== "bedrock-converse-stream") { + throw new BedrockConverseStreamCodecError( + `Model ${model.modelId} has no Bedrock Converse Stream compatibility record`, + ); + } + return model.compatibility; +} + +function requirePrepared(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new BedrockConverseStreamCodecError( + "Bedrock Converse Stream requires a prepared model request", + ); + } +} + +function nonEmpty(value: string | undefined, label: string): string { + const result = value?.trim(); + if (!result) throw new BedrockConverseStreamCodecError(`Bedrock ${label} is required`); + return result; +} + +function regionFromModelId(modelId: string): string | undefined { + const match = modelId.match(/^arn:aws(?:-[a-z0-9-]+)?:bedrock:([a-z0-9-]+):/); + return match?.[1]; +} + +function resolvedRegion(modelId: string, configured: string | undefined): string { + const region = regionFromModelId(modelId) ?? nonEmpty(configured, "region"); + if (!REGION_PATTERN.test(region)) { + throw new BedrockConverseStreamCodecError("Bedrock region is invalid"); + } + return region; +} + +function requestUrl(modelId: string, baseUrl: string | undefined, region: string): string { + let url: URL; + try { + url = new URL(baseUrl ?? `https://bedrock-runtime.${region}.amazonaws.com`); + } catch (cause) { + throw new BedrockConverseStreamCodecError("Bedrock base URL is invalid", { cause }); + } + if (url.protocol !== "https:" && url.protocol !== "http:") { + throw new BedrockConverseStreamCodecError("Bedrock base URL must use HTTP or HTTPS"); + } + if (url.username || url.password || url.hash) { + throw new BedrockConverseStreamCodecError("Bedrock base URL contains unsupported URL data"); + } + url.pathname = `${stripTrailingSlashes(url.pathname)}/model/${encodeURIComponent( + nonEmpty(modelId, "model ID"), + )}/converse-stream`; + return url.toString(); +} + +function headerValue(value: string, label: string): string { + const result = nonEmpty(value, label); + if (/\r|\n/.test(result)) { + throw new BedrockConverseStreamCodecError(`Bedrock ${label} contains invalid characters`); + } + return result; +} + +function imageFormat(mediaType: string): "jpeg" | "png" | "gif" | "webp" { + if (mediaType === "image/jpeg" || mediaType === "image/jpg") return "jpeg"; + if (mediaType === "image/png") return "png"; + if (mediaType === "image/gif") return "gif"; + if (mediaType === "image/webp") return "webp"; + throw new BedrockConverseStreamCodecError( + `Bedrock does not support image media type ${mediaType}`, + ); +} + +function contentBlock( + request: PreparedModelRequest, + content: Extract["content"][number], +): JsonObject | undefined { + if (content.type === "text") { + return content.text.trim().length === 0 ? undefined : { text: content.text }; + } + const blob = request.preparation.blobs.get(content.blob.sha256); + if (blob === undefined) { + throw new BedrockConverseStreamCodecError( + `Prepared blob ${content.blob.sha256} is unavailable`, + ); + } + return { + image: { + format: imageFormat(blob.reference.mediaType), + source: { bytes: Buffer.from(blob.bytes).toString("base64") }, + }, + }; +} + +function basicContent( + request: PreparedModelRequest, + message: Extract, +): JsonObject[] { + const result = message.content.flatMap((content) => { + const block = contentBlock(request, content); + return block === undefined ? [] : [block]; + }); + return result.length === 0 ? [{ text: EMPTY_TEXT_PLACEHOLDER }] : result; +} + +function sanitizeDocument(value: JsonValue): JsonValue { + if (Array.isArray(value)) return value.map(sanitizeDocument); + if (value !== null && typeof value === "object") { + return Object.fromEntries( + Object.entries(value) + .filter(([key]) => key.length > 0) + .map(([key, item]) => [key, sanitizeDocument(item)]), + ); + } + return value; +} + +function assistantContent( + model: ModelInfo, + message: Extract, + messageIndex: number, +): JsonObject[] { + const compat = compatibility(model); + if (message.continuation !== undefined) { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}] has continuation metadata unsupported by Bedrock`, + ); + } + const result: JsonObject[] = []; + for (const [contentIndex, content] of message.content.entries()) { + if (content.type === "blob") { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}].content[${contentIndex}] cannot replay an assistant image`, + ); + } + if (content.type === "text") { + if (content.continuation !== undefined) { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported continuation metadata`, + ); + } + if (content.text.trim().length > 0) result.push({ text: content.text }); + continue; + } + if (content.redacted === true) { + if (content.signature === undefined) { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}].content[${contentIndex}] has redacted reasoning without a signature`, + ); + } + result.push({ reasoningContent: { redactedContent: content.signature.value } }); + continue; + } + if (content.signature !== undefined && compat.supportsThinkingSignatures === true) { + result.push({ + reasoningContent: { + reasoningText: { text: content.text, signature: content.signature.value }, + }, + }); + } else if (compat.supportsThinkingSignatures === true) { + if (content.text.trim().length > 0) result.push({ text: content.text }); + } else if (content.text.trim().length > 0) { + result.push({ reasoningContent: { reasoningText: { text: content.text } } }); + } + } + for (const [callIndex, call] of (message.toolCalls ?? []).entries()) { + if (call.continuation !== undefined || call.signature !== undefined) { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported replay metadata`, + ); + } + result.push({ + toolUse: { toolUseId: call.callId, name: call.name, input: sanitizeDocument(call.input) }, + }); + } + if (result.length === 0) { + throw new BedrockConverseStreamCodecError( + `messages[${messageIndex}] has no Bedrock-renderable assistant content`, + ); + } + return result; +} + +function encodeMessages(model: ModelInfo, request: PreparedModelRequest): JsonObject[] { + const messages: MutableJsonObject[] = []; + for (let index = 0; index < request.messages.length; index += 1) { + const message = request.messages[index]; + if (message === undefined) continue; + if (message.role === "user") { + messages.push({ role: "user", content: basicContent(request, message) }); + continue; + } + if (message.role === "assistant") { + messages.push({ role: "assistant", content: assistantContent(model, message, index) }); + continue; + } + const results: JsonObject[] = []; + let resultIndex = index; + while (resultIndex < request.messages.length) { + const result = request.messages[resultIndex]; + if (result?.role !== "tool") break; + results.push({ + toolResult: { + toolUseId: result.callId, + content: basicContent(request, result), + status: result.isError ? "error" : "success", + }, + }); + resultIndex += 1; + } + messages.push({ role: "user", content: results }); + index = resultIndex - 1; + } + + if ( + request.preparation.cache.retention !== "none" && + compatibility(model).supportsPromptCacheMarkers === true + ) { + const lastUser = messages.findLast((message) => message.role === "user"); + const content = lastUser?.content; + if (Array.isArray(content)) { + content.push({ + cachePoint: { + type: "default", + ...(request.preparation.cache.retention === "long" ? { ttl: "1h" } : {}), + }, + }); + } + } + return messages; +} + +function encodeTools(request: PreparedModelRequest): JsonObject[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + return request.tools.map((tool) => { + if (tool.preparedConstraint?.type === "grammar") { + throw new BedrockConverseStreamCodecError( + `Bedrock cannot render grammar-constrained tool ${tool.canonicalName}`, + ); + } + return { + toolSpec: { + name: tool.name, + description: tool.description, + inputSchema: { json: tool.inputSchema }, + ...(tool.preparedConstraint?.type === "json-schema" && tool.preparedConstraint.strict + ? { strict: true } + : {}), + }, + }; + }); +} + +function additionalFields( + model: ModelInfo, + request: PreparedModelRequest, + policy: BedrockRequestPolicy, +): JsonObject | undefined { + const result: MutableJsonObject = {}; + for (const [key, value] of Object.entries(request.sampling?.custom ?? {})) { + if (RESERVED_REQUEST_FIELDS.has(key)) { + throw new BedrockConverseStreamCodecError( + `Custom sampling field ${key} collides with a Bedrock request field`, + ); + } + result[key] = value; + } + const reasoning = request.preparation.reasoning; + const compat = compatibility(model); + if ( + reasoning !== undefined && + reasoning.effective !== "off" && + compat.supportsThinkingSignatures === true + ) { + if ( + policy.thinkingDisplay !== undefined && + policy.thinkingDisplay !== "summarized" && + policy.thinkingDisplay !== "omitted" + ) { + throw new BedrockConverseStreamCodecError("Bedrock thinking display is invalid"); + } + const govCloud = + resolvedRegion(model.modelId, policy.region).startsWith("us-gov-") || + model.modelId.toLowerCase().startsWith("us-gov."); + const display = govCloud ? undefined : (policy.thinkingDisplay ?? "summarized"); + if (compat.forceAdaptiveThinking === true) { + result.thinking = { + type: "adaptive", + ...(display === undefined ? {} : { display }), + }; + result.output_config = { effort: reasoning.providerValue ?? reasoning.effective }; + } else { + if (reasoning.tokenBudget === undefined) { + throw new BedrockConverseStreamCodecError( + "Bedrock Claude thinking requires a prepared token budget", + ); + } + result.thinking = { + type: "enabled", + budget_tokens: reasoning.tokenBudget, + ...(display === undefined ? {} : { display }), + }; + if (policy.interleavedThinking !== false) { + result.anthropic_beta = ["interleaved-thinking-2025-05-14"]; + } + } + } + return Object.keys(result).length === 0 ? undefined : result; +} + +function requestMetadata(metadata: PreparedModelRequest["metadata"]): JsonObject | undefined { + if (metadata === undefined) return undefined; + const entries = Object.entries(metadata); + if (entries.length > 50) { + throw new BedrockConverseStreamCodecError("Bedrock request metadata exceeds 50 entries"); + } + const result: Record = {}; + for (const [key, value] of entries) { + if (key.length === 0 || key.length > 64 || key.toLowerCase().startsWith("aws:")) { + throw new BedrockConverseStreamCodecError(`Bedrock request metadata key ${key} is invalid`); + } + if (typeof value !== "string" || value.length > 256) { + throw new BedrockConverseStreamCodecError( + `Bedrock request metadata value for ${key} must be a string of at most 256 characters`, + ); + } + result[key] = value; + } + return result; +} + +/** Encodes a prepared request and the non-secret inputs required by AWS SigV4. */ +export function encodeBedrockConverseStreamRequest( + model: ModelInfo, + request: PreparedModelRequest, + policy: BedrockRequestPolicy, +): EncodedBedrockConverseStreamRequest { + requirePrepared(request); + compatibility(model); + if (request.modelId !== model.modelId) { + throw new BedrockConverseStreamCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (typeof policy !== "object" || policy === null) { + throw new BedrockConverseStreamCodecError("Bedrock request policy is invalid"); + } + const region = resolvedRegion(model.modelId, policy.region); + const body: MutableJsonObject = { messages: encodeMessages(model, request) }; + if (request.system !== undefined && request.system.trim().length > 0) { + body.system = [ + { text: request.system }, + ...(request.preparation.cache.retention !== "none" && + compatibility(model).supportsPromptCacheMarkers === true + ? [ + { + cachePoint: { + type: "default", + ...(request.preparation.cache.retention === "long" ? { ttl: "1h" } : {}), + }, + }, + ] + : []), + ]; + } + const inferenceConfig: MutableJsonObject = {}; + const maxTokens = + request.maxOutputTokens ?? + (compatibility(model).supportsThinkingSignatures === true ? model.maxOutputTokens : undefined); + if (maxTokens !== undefined) inferenceConfig.maxTokens = maxTokens; + if (request.sampling?.temperature !== undefined) + inferenceConfig.temperature = request.sampling.temperature; + if (request.sampling?.topP !== undefined) inferenceConfig.topP = request.sampling.topP; + if (Object.keys(inferenceConfig).length > 0) body.inferenceConfig = inferenceConfig; + const tools = request.toolChoice === "none" ? undefined : encodeTools(request); + if (tools !== undefined) { + body.toolConfig = { + tools, + ...(request.toolChoice === undefined + ? {} + : { + toolChoice: + request.toolChoice === "required" + ? { any: {} } + : request.toolChoice === "auto" + ? { auto: {} } + : {}, + }), + }; + } else if (request.toolChoice !== undefined && request.toolChoice !== "none") { + throw new BedrockConverseStreamCodecError( + `toolChoice ${request.toolChoice} needs at least one tool`, + ); + } + const extra = additionalFields(model, request, policy); + if (extra !== undefined) body.additionalModelRequestFields = extra; + const metadata = requestMetadata(request.metadata); + if (metadata !== undefined) body.requestMetadata = metadata; + + const headers: Record = { + accept: "application/vnd.amazon.eventstream", + "content-type": "application/json", + }; + let signing: AwsSigningInputs | undefined; + if (typeof policy.authentication !== "object" || policy.authentication === null) { + throw new BedrockConverseStreamCodecError("Bedrock authentication policy is invalid"); + } + if (policy.authentication.type === "bearer") { + headers.authorization = `Bearer ${headerValue(policy.authentication.token, "bearer token")}`; + } else if (policy.authentication.type === "sigv4") { + signing = { service: "bedrock", region }; + } else { + throw new BedrockConverseStreamCodecError("Bedrock authentication policy is invalid"); + } + return { + method: "POST", + url: requestUrl(model.modelId, policy.baseUrl, region), + body, + headers, + ...(signing === undefined ? {} : { signing }), + }; +} + +interface TextBlock { + readonly type: "text"; + readonly contentIndex: number; +} + +interface ThinkingBlock { + readonly type: "thinking"; + readonly contentIndex: number; + signature: string; + redacted: boolean; + redactedChunks: Uint8Array[]; +} + +interface ToolBlock { + readonly type: "tool"; + readonly contentIndex: number; + readonly callId: string; + readonly wireName: string; + argumentsText: string; +} + +type ActiveBlock = TextBlock | ThinkingBlock | ToolBlock; + +function eventIndex(event: Record, label: string): number { + if (!Number.isSafeInteger(event.contentBlockIndex) || (event.contentBlockIndex as number) < 0) { + throw new BedrockConverseStreamCodecError(`Bedrock ${label} has no valid content block index`); + } + return event.contentBlockIndex as number; +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function bytes(value: unknown): Uint8Array | undefined { + if (value instanceof Uint8Array) return value; + if ( + typeof value !== "string" || + value.length === 0 || + value.length % 4 !== 0 || + !/^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?$/.test(value) + ) { + return undefined; + } + return new Uint8Array(Buffer.from(value, "base64")); +} + +function base64(chunks: readonly Uint8Array[]): string { + return Buffer.concat(chunks.map((chunk) => Buffer.from(chunk))).toString("base64"); +} + +function nonNegative(value: unknown, label: string): number { + if (value === undefined || value === null) return 0; + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new BedrockConverseStreamCodecError(`Bedrock usage ${label} must be non-negative`); + } + return value; +} + +function mapUsage(raw: unknown, model: ModelInfo): Usage { + const value = object(raw); + if (value === undefined) + throw new BedrockConverseStreamCodecError("Bedrock usage must be an object"); + const mapped: Usage = { + inputTokens: nonNegative(value.inputTokens, "inputTokens"), + outputTokens: nonNegative(value.outputTokens, "outputTokens"), + cacheReadTokens: nonNegative(value.cacheReadInputTokens, "cacheReadInputTokens"), + cacheWriteTokens: nonNegative(value.cacheWriteInputTokens, "cacheWriteInputTokens"), + reasoningTokens: 0, + }; + return model.cost === undefined ? mapped : withUsageCost(model.cost, mapped); +} + +function stopReason(reason: string): "stop" | "length" | "tool_use" | "error" { + if (reason === "end_turn" || reason === "stop_sequence") return "stop"; + if (reason === "max_tokens" || reason === "model_context_window_exceeded") return "length"; + if (reason === "tool_use") return "tool_use"; + return "error"; +} + +function exception(event: Record): { code: string; value: unknown } | undefined { + for (const code of [ + "internalServerException", + "modelStreamErrorException", + "validationException", + "throttlingException", + "serviceUnavailableException", + ]) { + if (event[code] !== undefined) return { code, value: event[code] }; + } + return undefined; +} + +function finalizeBlock( + block: ActiveBlock, + options: BedrockConverseStreamDecodeOptions, +): readonly ModelStreamEvent[] { + if (block.type === "text") return []; + if (block.type === "thinking") { + const signature = block.redacted ? base64(block.redactedChunks) : block.signature; + return signature.length === 0 + ? [] + : [ + { + type: "replay_metadata", + target: "thinking", + contentIndex: block.contentIndex, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.request.modelId, + signature, + ...(block.redacted ? { redacted: true } : {}), + }, + ]; + } + let input: unknown; + try { + input = JSON.parse(block.argumentsText || "{}"); + } catch (cause) { + throw new BedrockConverseStreamCodecError("Bedrock tool input is not valid JSON", { cause }); + } + const parsed = object(input); + if (parsed === undefined) { + throw new BedrockConverseStreamCodecError("Bedrock tool input must be an object"); + } + return [ + { + type: "tool_call", + contentIndex: block.contentIndex, + callId: block.callId, + name: reverseToolName(options.request, block.wireName), + input: parsed as JsonObject, + }, + ]; +} + +/** Decodes AWS SDK Converse events into canonical model stream events. */ +export async function* decodeBedrockConverseStream( + events: AsyncIterable, + options: BedrockConverseStreamDecodeOptions, +): AsyncGenerator { + requirePrepared(options.request); + compatibility(options.model); + const blocks = new Map(); + let nextContentIndex = 0; + let emittedContent = false; + let emittedToolCall = false; + let nativeStopReason: string | undefined; + let finalUsage: Usage = { + inputTokens: 0, + outputTokens: 0, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + ...(options.model.cost === undefined ? {} : { costUsd: 0 }), + }; + let latencyMs: number | undefined; + const response = () => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(options.routedModelId === undefined ? {} : { routedModelId: options.routedModelId }), + ...(options.responseId === undefined ? {} : { responseId: options.responseId }), + ...(nativeStopReason === undefined ? {} : { nativeStopReason }), + ...(latencyMs === undefined + ? options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) } + : { latencyMs }), + }); + + for await (const raw of events) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + const event = object(raw); + if (event === undefined) { + throw new BedrockConverseStreamCodecError("Bedrock stream event must be an object"); + } + const failure = exception(event); + if (failure !== undefined) { + const details = object(failure.value); + const message = safeProviderMessage( + typeof details?.message === "string" + ? details.message + : "Bedrock reported a stream failure", + options.secretValues, + ); + const retryable = + failure.code === "throttlingException" || failure.code === "serviceUnavailableException"; + yield { + type: "error", + code: failure.code, + message, + retryable, + category: + failure.code === "throttlingException" + ? "rate_limit" + : failure.code === "serviceUnavailableException" + ? "overloaded" + : failure.code === "validationException" + ? "invalid_request" + : "provider_internal", + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: response(), + }; + return; + } + + const messageStart = object(event.messageStart); + if (messageStart !== undefined) { + if (messageStart.role !== "assistant") { + throw new BedrockConverseStreamCodecError("Bedrock message start role must be assistant"); + } + continue; + } + const start = object(event.contentBlockStart); + if (start !== undefined) { + const providerIndex = eventIndex(start, "content block start"); + if (blocks.has(providerIndex)) { + throw new BedrockConverseStreamCodecError( + `Bedrock content block ${providerIndex} started twice`, + ); + } + const toolUse = object(object(start.start)?.toolUse); + if ( + toolUse === undefined || + typeof toolUse.toolUseId !== "string" || + toolUse.toolUseId.length === 0 || + typeof toolUse.name !== "string" || + toolUse.name.length === 0 + ) { + throw new BedrockConverseStreamCodecError("Bedrock content block start is malformed"); + } + const block: ToolBlock = { + type: "tool", + contentIndex: nextContentIndex++, + callId: toolUse.toolUseId, + wireName: toolUse.name, + argumentsText: "", + }; + blocks.set(providerIndex, block); + emittedContent = true; + yield { + type: "tool_call_start", + contentIndex: block.contentIndex, + callId: block.callId, + name: reverseToolName(options.request, block.wireName), + }; + continue; + } + const deltaEvent = object(event.contentBlockDelta); + if (deltaEvent !== undefined) { + const providerIndex = eventIndex(deltaEvent, "content block delta"); + const delta = object(deltaEvent.delta); + if (delta === undefined) { + throw new BedrockConverseStreamCodecError("Bedrock content block delta is malformed"); + } + let block = blocks.get(providerIndex); + if (delta.text !== undefined) { + if (typeof delta.text !== "string") { + throw new BedrockConverseStreamCodecError("Bedrock text delta is malformed"); + } + if (block === undefined) { + block = { type: "text", contentIndex: nextContentIndex++ }; + blocks.set(providerIndex, block); + } + if (block.type !== "text") { + throw new BedrockConverseStreamCodecError("Bedrock text delta targets a non-text block"); + } + if (delta.text.length > 0) { + emittedContent = true; + yield { type: "text_delta", text: delta.text, contentIndex: block.contentIndex }; + } + continue; + } + const toolUse = object(delta.toolUse); + if (toolUse !== undefined) { + if (block?.type !== "tool" || typeof toolUse.input !== "string") { + throw new BedrockConverseStreamCodecError("Bedrock tool input delta is malformed"); + } + block.argumentsText += toolUse.input; + if (toolUse.input.length > 0) { + yield { + type: "tool_call_delta", + contentIndex: block.contentIndex, + callId: block.callId, + argumentsDelta: toolUse.input, + }; + } + continue; + } + const reasoning = object(delta.reasoningContent); + if (reasoning !== undefined) { + if (block === undefined) { + block = { + type: "thinking", + contentIndex: nextContentIndex++, + signature: "", + redacted: false, + redactedChunks: [], + }; + blocks.set(providerIndex, block); + } + if (block.type !== "thinking") { + throw new BedrockConverseStreamCodecError( + "Bedrock reasoning delta targets a non-thinking block", + ); + } + if (reasoning.text !== undefined) { + if (typeof reasoning.text !== "string") { + throw new BedrockConverseStreamCodecError("Bedrock reasoning text is malformed"); + } + if (reasoning.text.length > 0) { + emittedContent = true; + yield { + type: "thinking_delta", + text: reasoning.text, + contentIndex: block.contentIndex, + }; + } + } + if (reasoning.signature !== undefined) { + if (typeof reasoning.signature !== "string") { + throw new BedrockConverseStreamCodecError("Bedrock reasoning signature is malformed"); + } + if (!block.redacted) block.signature += reasoning.signature; + } + if (reasoning.redactedContent !== undefined) { + const chunk = bytes(reasoning.redactedContent); + if (chunk === undefined || chunk.length === 0) { + throw new BedrockConverseStreamCodecError("Bedrock redacted reasoning is malformed"); + } + if (!block.redacted) { + block.redacted = true; + block.signature = ""; + emittedContent = true; + yield { + type: "thinking_delta", + text: REDACTED_THINKING_PLACEHOLDER, + contentIndex: block.contentIndex, + }; + } + block.redactedChunks.push(chunk); + } + continue; + } + continue; + } + const stop = object(event.contentBlockStop); + if (stop !== undefined) { + const providerIndex = eventIndex(stop, "content block stop"); + const block = blocks.get(providerIndex); + if (block === undefined) { + throw new BedrockConverseStreamCodecError( + `Bedrock stop targets unknown block ${providerIndex}`, + ); + } + for (const finalized of finalizeBlock(block, options)) { + if (finalized.type === "tool_call") emittedToolCall = true; + yield finalized; + } + blocks.delete(providerIndex); + continue; + } + const messageStop = object(event.messageStop); + if (messageStop !== undefined) { + if (typeof messageStop.stopReason !== "string" || messageStop.stopReason.length === 0) { + throw new BedrockConverseStreamCodecError("Bedrock message stop reason is malformed"); + } + nativeStopReason = messageStop.stopReason; + continue; + } + const metadata = object(event.metadata); + if (metadata !== undefined) { + if (metadata.usage !== undefined) finalUsage = mapUsage(metadata.usage, options.model); + const metrics = object(metadata.metrics); + if (metrics?.latencyMs !== undefined) { + latencyMs = nonNegative(metrics.latencyMs, "latencyMs"); + } + } + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (nativeStopReason === undefined) return; + for (const [, block] of [...blocks.entries()].sort(([left], [right]) => left - right)) { + for (const finalized of finalizeBlock(block, options)) { + if (finalized.type === "tool_call") emittedToolCall = true; + yield finalized; + } + } + const mapped = stopReason(nativeStopReason); + if (mapped === "error") { + yield { + type: "error", + code: nativeStopReason, + message: safeProviderMessage( + `Provider stopped with: ${nativeStopReason}`, + options.secretValues, + ), + retryable: false, + category: + nativeStopReason === "guardrail_intervened" || nativeStopReason === "content_filtered" + ? "content_policy" + : "provider_internal", + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: response(), + }; + return; + } + yield { + type: "completed", + stopReason: mapped === "tool_use" || emittedToolCall ? "tool_use" : mapped, + usage: finalUsage, + ...(mapped === "length" ? { partial: true } : {}), + response: response(), + }; +} diff --git a/packages/ai/src/builtin-providers.ts b/packages/ai/src/builtin-providers.ts new file mode 100644 index 0000000..eaef5b6 --- /dev/null +++ b/packages/ai/src/builtin-providers.ts @@ -0,0 +1,146 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createAntLingProvider } from "./ant-ling.ts"; +import { createBasetenProvider } from "./baseten.ts"; +import { enableStaticCatalogRefresh } from "./catalog-refresh.ts"; +import { createCerebrasProvider } from "./cerebras.ts"; +import { createDeepSeekProvider } from "./deepseek.ts"; +import { createFireworksProvider } from "./fireworks.ts"; +import { createGroqProvider } from "./groq.ts"; +import { createHuggingFaceProvider } from "./huggingface.ts"; +import { createMiniMaxProvider } from "./minimax.ts"; +import { createMiniMaxCnProvider } from "./minimax-cn.ts"; +import { createMoonshotAiProvider } from "./moonshotai.ts"; +import { createMoonshotAiCnProvider } from "./moonshotai-cn.ts"; +import { createNvidiaProvider } from "./nvidia.ts"; +import type { ModelProvider } from "./provider.ts"; +import { createQwenTokenPlanProvider } from "./qwen-token-plan.ts"; +import { createQwenTokenPlanCnProvider } from "./qwen-token-plan-cn.ts"; +import { createQwenTokenPlanIndividualProvider } from "./qwen-token-plan-individual.ts"; +import { + type CustomProviderConfiguration, + createAmazonBedrockProvider, + createAnthropicProvider, + createAzureOpenAiResponsesProvider, + createCloudflareAiGatewayProvider, + createCloudflareWorkersAiProvider, + createCustomProvider, + createGitHubCopilotProvider, + createGoogleProvider, + createGoogleVertexProvider, + createKimiCodingProvider, + createMistralProvider, + createOpenAiCodexProvider, + createOpenAiProvider, + createOpenCodeGoProvider, + createOpenCodeProvider, + createOpenRouterProvider, + createRadiusProvider, + type ProviderFactoryOptions, +} from "./remaining-providers.ts"; +import { createTogetherProvider } from "./together.ts"; +import { createVercelAiGatewayProvider } from "./vercel-ai-gateway.ts"; +import { createXaiProvider } from "./xai.ts"; +import { createXiaomiProvider } from "./xiaomi.ts"; +import { createXiaomiTokenPlanAmsProvider } from "./xiaomi-token-plan-ams.ts"; +import { createXiaomiTokenPlanCnProvider } from "./xiaomi-token-plan-cn.ts"; +import { createXiaomiTokenPlanSgpProvider } from "./xiaomi-token-plan-sgp.ts"; +import { createZaiProvider } from "./zai.ts"; +import { createZaiCodingCnProvider } from "./zai-coding-cn.ts"; + +export const BUILTIN_PROVIDER_IDS = [ + "openai", + "azure-openai-responses", + "openai-codex", + "anthropic", + "google", + "google-vertex", + "amazon-bedrock", + "github-copilot", + "xai", + "deepseek", + "mistral", + "groq", + "cerebras", + "nvidia", + "openrouter", + "vercel-ai-gateway", + "cloudflare-ai-gateway", + "cloudflare-workers-ai", + "fireworks", + "together", + "baseten", + "huggingface", + "zai", + "zai-coding-cn", + "minimax", + "minimax-cn", + "moonshotai", + "moonshotai-cn", + "kimi-coding", + "qwen-token-plan", + "qwen-token-plan-individual", + "qwen-token-plan-cn", + "xiaomi", + "xiaomi-token-plan-cn", + "xiaomi-token-plan-ams", + "xiaomi-token-plan-sgp", + "opencode", + "opencode-go", + "ant-ling", + "radius", + "custom", +] as const; + +/** Constructs every planned built in registration without I/O. */ +export function createBuiltinProviders( + options: ProviderFactoryOptions, + custom?: CustomProviderConfiguration, +): readonly ModelProvider[] { + const providers = [ + createOpenAiProvider(options), + createAzureOpenAiResponsesProvider(options), + createOpenAiCodexProvider(options), + createAnthropicProvider(options), + createGoogleProvider(options), + createGoogleVertexProvider(options), + createAmazonBedrockProvider(options), + createGitHubCopilotProvider(options), + createXaiProvider(options), + createDeepSeekProvider(options), + createMistralProvider(options), + createGroqProvider(options), + createCerebrasProvider(options), + createNvidiaProvider(options), + createOpenRouterProvider(options), + createVercelAiGatewayProvider(options), + createCloudflareAiGatewayProvider(options), + createCloudflareWorkersAiProvider(options), + createFireworksProvider(options), + createTogetherProvider(options), + createBasetenProvider(options), + createHuggingFaceProvider(options), + createZaiProvider(options), + createZaiCodingCnProvider(options), + createMiniMaxProvider(options), + createMiniMaxCnProvider(options), + createMoonshotAiProvider(options), + createMoonshotAiCnProvider(options), + createKimiCodingProvider(options), + createQwenTokenPlanProvider(options), + createQwenTokenPlanIndividualProvider(options), + createQwenTokenPlanCnProvider(options), + createXiaomiProvider(options), + createXiaomiTokenPlanCnProvider(options), + createXiaomiTokenPlanAmsProvider(options), + createXiaomiTokenPlanSgpProvider(options), + createOpenCodeProvider(options), + createOpenCodeGoProvider(options), + createAntLingProvider(options), + createRadiusProvider(options), + createCustomProvider({ ...options, ...custom }), + ]; + for (const provider of providers) enableStaticCatalogRefresh(provider, options.fetch); + return providers; +} diff --git a/packages/ai/src/catalog-normalization.ts b/packages/ai/src/catalog-normalization.ts new file mode 100644 index 0000000..74e13ac --- /dev/null +++ b/packages/ai/src/catalog-normalization.ts @@ -0,0 +1,259 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { ThinkingLevel } from "@axl/protocol"; +import type { ProviderCatalogOverlay } from "./catalog-overlays.ts"; +import { validateModelCatalog } from "./catalog-validation.ts"; +import type { KnownApiDialect, ModelAvailability, ModelCost, ModelInfo } from "./model.ts"; + +interface SourceReasoningOption { + readonly type?: unknown; + readonly values?: unknown; +} + +export interface SourceModel { + readonly id?: unknown; + readonly name?: unknown; + readonly toolCall?: unknown; + readonly structuredOutput?: unknown; + readonly imageInput?: unknown; + readonly reasoning?: unknown; + readonly reasoningOptions?: unknown; + readonly contextWindow?: unknown; + readonly maxOutputTokens?: unknown; + readonly cost?: unknown; + readonly status?: unknown; +} + +export function object(value: unknown, label: string): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error(`${label} must be an object`); + } + return value as Record; +} + +export function string(value: unknown, label: string): string { + if (typeof value !== "string" || value.trim().length === 0) { + throw new Error(`${label} must be a nonempty string`); + } + return value; +} + +export function positiveInteger(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) <= 0) { + throw new Error(`${label} must be a positive safe integer`); + } + return value as number; +} + +function optionalRate(value: unknown, label: string): number | undefined { + if (value === undefined) return undefined; + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new Error(`${label} must be a nonnegative finite number`); + } + return value; +} + +function sourceCost(value: unknown, label: string): ModelCost | undefined { + if (value === undefined) return undefined; + const input = object(value, label); + const inputRate = optionalRate(input.input, `${label}.input`) ?? 0; + const outputRate = optionalRate(input.output, `${label}.output`) ?? 0; + const cacheRead = optionalRate(input.cache_read, `${label}.cache_read`); + const cacheWrite = optionalRate(input.cache_write, `${label}.cache_write`); + const rawTiers = input.tiers; + const tiers = Array.isArray(rawTiers) + ? rawTiers + .map((entry, index) => { + const tier = object(entry, `${label}.tiers[${index}]`); + const condition = object(tier.tier, `${label}.tiers[${index}].tier`); + if (condition.type !== "context") return undefined; + const tierCacheRead = optionalRate( + tier.cache_read, + `${label}.tiers[${index}].cache_read`, + ); + const tierCacheWrite = optionalRate( + tier.cache_write, + `${label}.tiers[${index}].cache_write`, + ); + return { + inputTokensAbove: positiveInteger(condition.size, `${label}.tiers[${index}].tier.size`), + inputUsdPerMTok: optionalRate(tier.input, `${label}.tiers[${index}].input`) ?? 0, + outputUsdPerMTok: optionalRate(tier.output, `${label}.tiers[${index}].output`) ?? 0, + ...(tierCacheRead === undefined ? {} : { cacheReadUsdPerMTok: tierCacheRead }), + ...(tierCacheWrite === undefined ? {} : { cacheWriteUsdPerMTok: tierCacheWrite }), + }; + }) + .filter((tier) => tier !== undefined) + .sort((left, right) => left.inputTokensAbove - right.inputTokensAbove) + : []; + return { + inputUsdPerMTok: inputRate, + outputUsdPerMTok: outputRate, + ...(cacheRead === undefined ? {} : { cacheReadUsdPerMTok: cacheRead }), + ...(cacheWrite === undefined ? {} : { cacheWriteUsdPerMTok: cacheWrite }), + ...(tiers.length === 0 ? {} : { tiers }), + }; +} + +function reasoningMap(model: SourceModel, label: string): ModelInfo["thinkingLevelMap"] { + if (model.reasoning !== true) return undefined; + if (model.reasoningOptions === undefined) return undefined; + if (!Array.isArray(model.reasoningOptions)) { + throw new Error(`${label}.reasoningOptions must be an array`); + } + const options = model.reasoningOptions.map((value, index) => + object(value, `${label}.reasoningOptions[${index}]`), + ) as SourceReasoningOption[]; + const effort = options.find((option) => option.type === "effort"); + if (effort !== undefined) { + if (!Array.isArray(effort.values) || effort.values.some((value) => typeof value !== "string")) { + throw new Error(`${label} has invalid effort values`); + } + const values = new Set(effort.values as string[]); + const map: Partial> = {}; + for (const level of ["minimal", "low", "medium", "high", "xhigh", "max"] as const) { + map[level] = values.has(level) ? level : null; + } + if (values.has("none")) map.off = "none"; + else if (values.has("off")) map.off = "off"; + return map; + } + if (options.some((option) => option.type === "budget_tokens")) { + return { + minimal: "1024", + low: "2048", + medium: "8192", + high: "16384", + xhigh: "16384", + max: "16384", + }; + } + if (options.some((option) => option.type === "toggle")) { + return { off: "disabled", minimal: null, low: null, medium: null, high: "enabled" }; + } + return undefined; +} + +function availability(status: unknown, label: string): ModelAvailability { + if (status === undefined || status === "active") return { status: "available" }; + if (status === "alpha" || status === "beta" || status === "preview") { + return { status: "preview", reason: `Source catalog status: ${status}` }; + } + if (status === "deprecated") { + return { status: "deprecated", reason: "Deprecated by the source catalog" }; + } + throw new Error(`${label}.status has unsupported value ${JSON.stringify(status)}`); +} + +function dialectFor(overlay: ProviderCatalogOverlay, modelId: string): KnownApiDialect { + for (const rule of overlay.dialectRules ?? []) { + if (modelId.startsWith(rule.prefix)) return rule.dialect; + } + if (overlay.dialect === undefined) throw new Error(`${overlay.id} has no default dialect`); + return overlay.dialect; +} + +function selected(source: NonNullable, modelId: string): boolean { + if ( + source.includePrefixes && + !source.includePrefixes.some((prefix) => modelId.startsWith(prefix)) + ) { + return false; + } + if (source.excludeModelIds?.includes(modelId)) return false; + return !source.excludeSuffixes?.some((suffix) => modelId.endsWith(suffix)); +} + +function normalizeModel( + overlay: ProviderCatalogOverlay, + modelId: string, + value: unknown, +): ModelInfo { + const label = `${overlay.source?.providerId}/${modelId}`; + const source = object(value, label) as SourceModel; + if (source.id !== modelId) throw new Error(`${label} source identity does not match its key`); + const dialect = dialectFor(overlay, modelId); + const contextWindow = positiveInteger(source.contextWindow, `${label}.contextWindow`); + const maxOutputTokens = positiveInteger(source.maxOutputTokens, `${label}.maxOutputTokens`); + if (maxOutputTokens > contextWindow) { + throw new Error(`${label} output limit exceeds its context window`); + } + const thinkingLevelMap = reasoningMap(source, label); + const cost = sourceCost(source.cost, `${label}.cost`); + const baseCompatibility = overlay.compatibilityByDialect?.[dialect]; + let compatibility = baseCompatibility; + if ( + baseCompatibility?.dialect === "anthropic-messages" && + overlay.anthropicAdaptiveThinkingPrefixes?.some((prefix) => modelId.startsWith(prefix)) + ) { + compatibility = { ...baseCompatibility, forceAdaptiveThinking: true }; + } + if ( + (baseCompatibility?.dialect === "google-generative-ai" || + baseCompatibility?.dialect === "google-vertex") && + overlay.googleStrictToolPrefixes?.some((prefix) => modelId.startsWith(prefix)) + ) { + compatibility = { ...baseCompatibility, supportsStrictTools: true }; + } + if (baseCompatibility?.dialect === "bedrock-converse-stream") { + const isClaude = modelId.includes("anthropic.claude"); + const adaptive = + isClaude && + ["opus-4-6", "opus-4-7", "opus-4-8", "opus-5", "sonnet-4-6", "sonnet-5", "fable-5"].some( + (name) => modelId.includes(name), + ); + compatibility = { + ...baseCompatibility, + ...(source.structuredOutput === true ? { supportsStrictTools: true } : {}), + ...(isClaude + ? { + supportsPromptCacheMarkers: true, + supportsThinkingSignatures: true, + ...(adaptive ? { forceAdaptiveThinking: true } : {}), + } + : {}), + }; + } + if (baseCompatibility?.dialect === "mistral-conversations") { + compatibility = { + ...baseCompatibility, + ...(source.structuredOutput === true ? { supportsStrictTools: true } : {}), + }; + } + return { + providerId: overlay.id, + modelId, + displayName: string(source.name, `${label}.name`).trim(), + apiDialect: dialect, + capabilities: { + toolUse: source.toolCall === true, + structuredOutput: source.structuredOutput === true, + imageInput: source.imageInput === true, + }, + reasoning: source.reasoning === true, + ...(thinkingLevelMap === undefined ? {} : { thinkingLevelMap }), + contextWindow, + maxOutputTokens, + ...(cost === undefined ? {} : { cost }), + ...(overlay.cache === undefined ? {} : { cache: overlay.cache }), + ...(overlay.endpoint === undefined ? {} : { endpoint: overlay.endpoint }), + availability: availability(source.status, label), + ...(compatibility === undefined ? {} : { compatibility }), + }; +} + +export function normalizeCatalogModels( + overlay: ProviderCatalogOverlay, + models: Record, +): readonly ModelInfo[] { + const source = overlay.source; + if (!source) throw new Error(`${overlay.id} has no catalog source`); + const normalized = Object.entries(models) + .filter(([id, model]) => object(model, id).toolCall === true && selected(source, id)) + .map(([id, model]) => normalizeModel(overlay, id, model)) + .sort((left, right) => left.modelId.localeCompare(right.modelId)); + if (normalized.length === 0) throw new Error(`${overlay.id} returned an empty catalog`); + validateModelCatalog(normalized); + return normalized; +} diff --git a/packages/ai/src/catalog-overlays.ts b/packages/ai/src/catalog-overlays.ts new file mode 100644 index 0000000..1585a09 --- /dev/null +++ b/packages/ai/src/catalog-overlays.ts @@ -0,0 +1,664 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { + EndpointPolicy, + KnownApiDialect, + ModelCachePolicy, + ModelCompatibility, +} from "./model.ts"; + +export type CatalogKind = "static" | "dynamic" | "configured"; + +export interface CatalogDialectRule { + readonly prefix: string; + readonly dialect: KnownApiDialect; +} + +export interface CatalogSource { + readonly manifest: "models-dev" | "ant-ling"; + readonly providerId: string; + readonly includePrefixes?: readonly string[]; + readonly excludeSuffixes?: readonly string[]; + readonly excludeModelIds?: readonly string[]; +} + +export interface ProviderCatalogOverlay { + readonly id: string; + readonly displayName: string; + readonly catalogKind: CatalogKind; + readonly source?: CatalogSource; + readonly dialect?: KnownApiDialect; + readonly dialectRules?: readonly CatalogDialectRule[]; + readonly endpoint?: EndpointPolicy; + readonly cache?: ModelCachePolicy; + readonly compatibilityByDialect?: Readonly>>; + /** Anthropic model prefixes that require adaptive rather than token-budget thinking. */ + readonly anthropicAdaptiveThinkingPrefixes?: readonly string[]; + /** Google model prefixes that support validated strict function calling. */ + readonly googleStrictToolPrefixes?: readonly string[]; + readonly regionFamily?: string; + readonly region?: string; +} + +const noCache = { + supported: false, + defaultRetention: "none", + supportedRetentions: ["none"], +} as const; + +const shortCache = { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], +} as const; + +const longCache = { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short", "long"], +} as const; + +const openAiChatCompatibility = { + dialect: "openai-chat", + supportsStore: false, + supportsDeveloperRole: false, + supportsReasoningEffort: false, + supportsUsageInStreaming: true, + maxTokensField: "max_tokens", + supportsStrictTools: false, + supportsLongCacheRetention: false, +} as const; + +const fixed = (baseUrl: string): EndpointPolicy => ({ type: "fixed", baseUrl }); + +export const PROVIDER_CATALOG_OVERLAYS: readonly ProviderCatalogOverlay[] = [ + { + id: "openai", + displayName: "OpenAI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "openai" }, + dialect: "openai-responses", + dialectRules: [ + { prefix: "gpt-3.5", dialect: "openai-chat" }, + { prefix: "gpt-4", dialect: "openai-chat" }, + ], + endpoint: fixed("https://api.openai.com/v1"), + cache: longCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, supportsDeveloperRole: true }, + "openai-responses": { + dialect: "openai-responses", + supportsDeveloperRole: true, + supportsStrictTools: true, + supportsGrammarTools: true, + supportsLongCacheRetention: true, + supportsMaxOutputTokens: true, + }, + }, + }, + { + id: "azure-openai-responses", + displayName: "Azure OpenAI Responses", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "azure" }, + dialect: "azure-openai-responses", + endpoint: { + type: "template", + template: "https://{resource}.openai.azure.com/openai/v1", + variables: [{ name: "resource", setting: "resource", required: true }], + }, + cache: shortCache, + compatibilityByDialect: { + "azure-openai-responses": { + dialect: "azure-openai-responses", + supportsDeveloperRole: true, + supportsStrictTools: true, + supportsGrammarTools: true, + supportsMaxOutputTokens: true, + }, + }, + }, + { + id: "openai-codex", + displayName: "OpenAI Codex", + catalogKind: "static", + source: { + manifest: "models-dev", + providerId: "openai", + includePrefixes: ["gpt-5"], + excludeSuffixes: ["chat-latest"], + }, + dialect: "openai-codex-responses", + endpoint: fixed("https://chatgpt.com/backend-api/codex"), + cache: longCache, + compatibilityByDialect: { + "openai-codex-responses": { + dialect: "openai-codex-responses", + supportsDeveloperRole: true, + supportsStrictTools: true, + supportsGrammarTools: true, + supportsLongCacheRetention: true, + supportsMaxOutputTokens: true, + }, + }, + }, + { + id: "anthropic", + displayName: "Anthropic", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "anthropic" }, + dialect: "anthropic-messages", + endpoint: fixed("https://api.anthropic.com"), + cache: longCache, + anthropicAdaptiveThinkingPrefixes: [ + "claude-fable-5", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-5", + ], + compatibilityByDialect: { + "anthropic-messages": { + dialect: "anthropic-messages", + supportsLongCacheRetention: true, + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + }, + }, + }, + { + id: "google", + displayName: "Google Generative AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "google" }, + dialect: "google-generative-ai", + endpoint: fixed("https://generativelanguage.googleapis.com/v1beta"), + cache: shortCache, + googleStrictToolPrefixes: ["gemini-3"], + compatibilityByDialect: { + "google-generative-ai": { + dialect: "google-generative-ai", + }, + }, + }, + { + id: "google-vertex", + displayName: "Google Vertex AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "google-vertex" }, + dialect: "google-vertex", + endpoint: { + type: "template", + template: + "https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}", + variables: [ + { name: "location", setting: "location", required: true }, + { name: "project", setting: "project", required: true }, + ], + }, + cache: shortCache, + googleStrictToolPrefixes: ["gemini-3"], + compatibilityByDialect: { + "google-vertex": { + dialect: "google-vertex", + }, + }, + }, + { + id: "amazon-bedrock", + displayName: "Amazon Bedrock", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "amazon-bedrock" }, + dialect: "bedrock-converse-stream", + endpoint: { + type: "template", + template: "https://bedrock-runtime.{region}.amazonaws.com", + variables: [{ name: "region", setting: "region", required: true }], + }, + cache: shortCache, + compatibilityByDialect: { + "bedrock-converse-stream": { dialect: "bedrock-converse-stream" }, + }, + }, + { + id: "github-copilot", + displayName: "GitHub Copilot", + catalogKind: "dynamic", + dialect: "openai-chat", + dialectRules: [{ prefix: "claude-", dialect: "anthropic-messages" }], + endpoint: fixed("https://api.githubcopilot.com"), + cache: shortCache, + }, + { + id: "xai", + displayName: "xAI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "xai" }, + dialect: "openai-chat", + endpoint: fixed("https://api.x.ai/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "deepseek", + displayName: "DeepSeek", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "deepseek" }, + dialect: "openai-chat", + endpoint: fixed("https://api.deepseek.com"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { + ...openAiChatCompatibility, + requiresReasoningContentOnAssistantMessages: true, + thinkingFormat: "deepseek", + }, + }, + }, + { + id: "mistral", + displayName: "Mistral", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "mistral" }, + dialect: "mistral-conversations", + endpoint: fixed("https://api.mistral.ai/v1"), + cache: shortCache, + compatibilityByDialect: { + "mistral-conversations": { dialect: "mistral-conversations" }, + }, + }, + { + id: "groq", + displayName: "Groq", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "groq" }, + dialect: "openai-chat", + endpoint: fixed("https://api.groq.com/openai/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "cerebras", + displayName: "Cerebras", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "cerebras" }, + dialect: "openai-chat", + endpoint: fixed("https://api.cerebras.ai/v1"), + cache: noCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "nvidia", + displayName: "NVIDIA NIM", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "nvidia" }, + dialect: "openai-chat", + endpoint: fixed("https://integrate.api.nvidia.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "openrouter", + displayName: "OpenRouter", + catalogKind: "dynamic", + dialect: "openai-chat", + endpoint: fixed("https://openrouter.ai/api/v1"), + cache: longCache, + }, + { + id: "vercel-ai-gateway", + displayName: "Vercel AI Gateway", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "vercel" }, + dialect: "openai-chat", + endpoint: fixed("https://ai-gateway.vercel.sh/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "cloudflare-ai-gateway", + displayName: "Cloudflare AI Gateway", + catalogKind: "dynamic", + dialect: "openai-chat", + endpoint: { + type: "template", + template: "https://gateway.ai.cloudflare.com/v1/{account}/{gateway}", + variables: [ + { name: "account", setting: "account", required: true }, + { name: "gateway", setting: "gateway", required: true }, + ], + }, + cache: shortCache, + }, + { + id: "cloudflare-workers-ai", + displayName: "Cloudflare Workers AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "cloudflare-workers-ai" }, + dialect: "openai-chat", + endpoint: { + type: "template", + template: "https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1", + variables: [{ name: "account", setting: "account", required: true }], + }, + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "fireworks", + displayName: "Fireworks AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "fireworks-ai" }, + dialect: "openai-chat", + endpoint: fixed("https://api.fireworks.ai/inference/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "together", + displayName: "Together AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "togetherai" }, + dialect: "openai-chat", + endpoint: fixed("https://api.together.ai/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "together" }, + }, + }, + { + id: "baseten", + displayName: "Baseten", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "baseten" }, + dialect: "openai-chat", + endpoint: fixed("https://inference.baseten.co/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "baseten" }, + }, + }, + { + id: "huggingface", + displayName: "Hugging Face", + catalogKind: "static", + source: { + manifest: "models-dev", + providerId: "huggingface", + excludeModelIds: ["thinkingmachines/Inkling-Small"], + }, + dialect: "openai-chat", + endpoint: fixed("https://router.huggingface.co/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "zai", + displayName: "Z.AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "zai" }, + dialect: "openai-chat", + endpoint: fixed("https://api.z.ai/api/paas/v4"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "zai" }, + }, + regionFamily: "zai", + region: "global", + }, + { + id: "zai-coding-cn", + displayName: "Z.AI Coding China", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "zhipuai-coding-plan" }, + dialect: "openai-chat", + endpoint: fixed("https://open.bigmodel.cn/api/coding/paas/v4"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "zai" }, + }, + regionFamily: "zai", + region: "cn", + }, + { + id: "minimax", + displayName: "MiniMax", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "minimax" }, + dialect: "openai-chat", + endpoint: fixed("https://api.minimax.io/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "minimax", + region: "global", + }, + { + id: "minimax-cn", + displayName: "MiniMax China", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "minimax-cn" }, + dialect: "openai-chat", + endpoint: fixed("https://api.minimaxi.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "minimax", + region: "cn", + }, + { + id: "moonshotai", + displayName: "Moonshot AI", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "moonshotai" }, + dialect: "openai-chat", + endpoint: fixed("https://api.moonshot.ai/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "moonshotai", + region: "global", + }, + { + id: "moonshotai-cn", + displayName: "Moonshot AI China", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "moonshotai-cn" }, + dialect: "openai-chat", + endpoint: fixed("https://api.moonshot.cn/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "moonshotai", + region: "cn", + }, + { + id: "kimi-coding", + displayName: "Kimi For Coding", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "kimi-for-coding" }, + dialect: "openai-chat", + endpoint: fixed("https://api.kimi.com/coding/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + }, + { + id: "qwen-token-plan", + displayName: "Qwen Token Plan", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "alibaba-token-plan" }, + dialect: "openai-chat", + endpoint: fixed("https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "qwen" }, + }, + regionFamily: "qwen-token-plan", + region: "sgp", + }, + { + id: "qwen-token-plan-individual", + displayName: "Qwen Token Plan Individual", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "alibaba-token-plan" }, + dialect: "openai-chat", + endpoint: fixed("https://coding-intl.dashscope.aliyuncs.com/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "qwen" }, + }, + }, + { + id: "qwen-token-plan-cn", + displayName: "Qwen Token Plan China", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "alibaba-token-plan-cn" }, + dialect: "openai-chat", + endpoint: fixed("https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "qwen" }, + }, + regionFamily: "qwen-token-plan", + region: "cn", + }, + { + id: "xiaomi", + displayName: "Xiaomi MiMo", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "xiaomi" }, + dialect: "openai-chat", + endpoint: fixed("https://api.xiaomimimo.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "xiaomi", + region: "global", + }, + { + id: "xiaomi-token-plan-cn", + displayName: "Xiaomi Token Plan China", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "xiaomi-token-plan-cn" }, + dialect: "openai-chat", + endpoint: fixed("https://token-plan-cn.xiaomimimo.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "xiaomi", + region: "cn", + }, + { + id: "xiaomi-token-plan-ams", + displayName: "Xiaomi Token Plan Amsterdam", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "xiaomi-token-plan-ams" }, + dialect: "openai-chat", + endpoint: fixed("https://token-plan-ams.xiaomimimo.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "xiaomi", + region: "ams", + }, + { + id: "xiaomi-token-plan-sgp", + displayName: "Xiaomi Token Plan Singapore", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "xiaomi-token-plan-sgp" }, + dialect: "openai-chat", + endpoint: fixed("https://token-plan-sgp.xiaomimimo.com/v1"), + cache: shortCache, + compatibilityByDialect: { "openai-chat": openAiChatCompatibility }, + regionFamily: "xiaomi", + region: "sgp", + }, + { + id: "opencode", + displayName: "OpenCode Zen", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "opencode" }, + dialect: "openai-chat", + dialectRules: [ + { prefix: "claude-", dialect: "anthropic-messages" }, + { prefix: "qwen3", dialect: "anthropic-messages" }, + { prefix: "gemini-", dialect: "google-generative-ai" }, + { prefix: "gpt-", dialect: "openai-responses" }, + { prefix: "grok-", dialect: "openai-responses" }, + { prefix: "muse-", dialect: "openai-responses" }, + ], + endpoint: fixed("https://opencode.ai/zen/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": openAiChatCompatibility, + "openai-responses": { + dialect: "openai-responses", + supportsDeveloperRole: true, + supportsStrictTools: true, + supportsGrammarTools: true, + supportsMaxOutputTokens: true, + }, + "anthropic-messages": { + dialect: "anthropic-messages", + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + }, + "google-generative-ai": { dialect: "google-generative-ai" }, + }, + }, + { + id: "opencode-go", + displayName: "OpenCode Go", + catalogKind: "static", + source: { manifest: "models-dev", providerId: "opencode-go" }, + dialect: "openai-chat", + dialectRules: [ + { prefix: "minimax-", dialect: "anthropic-messages" }, + { prefix: "qwen3", dialect: "anthropic-messages" }, + { prefix: "gpt-", dialect: "openai-responses" }, + { prefix: "grok-", dialect: "openai-responses" }, + { prefix: "muse-", dialect: "openai-responses" }, + ], + endpoint: fixed("https://opencode.ai/zen/go/v1"), + cache: shortCache, + compatibilityByDialect: { + "openai-chat": openAiChatCompatibility, + "openai-responses": { + dialect: "openai-responses", + supportsDeveloperRole: true, + supportsStrictTools: true, + supportsGrammarTools: true, + supportsMaxOutputTokens: true, + }, + "anthropic-messages": { + dialect: "anthropic-messages", + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + }, + }, + }, + { + id: "ant-ling", + displayName: "Ant Ling", + catalogKind: "static", + source: { manifest: "ant-ling", providerId: "ant-ling" }, + dialect: "openai-chat", + endpoint: fixed("https://api.ant-ling.com/v1"), + cache: noCache, + compatibilityByDialect: { + "openai-chat": { ...openAiChatCompatibility, thinkingFormat: "ant-ling" }, + }, + }, + { + id: "radius", + displayName: "Radius", + catalogKind: "dynamic", + dialect: "gateway-messages", + endpoint: { type: "configured", baseUrlSetting: "baseUrl" }, + cache: shortCache, + }, + { + id: "custom", + displayName: "User configured endpoint", + catalogKind: "configured", + dialect: "openai-chat", + endpoint: { type: "configured", baseUrlSetting: "baseUrl" }, + cache: noCache, + }, +] as const; diff --git a/packages/ai/src/catalog-refresh.ts b/packages/ai/src/catalog-refresh.ts new file mode 100644 index 0000000..7c8a152 --- /dev/null +++ b/packages/ai/src/catalog-refresh.ts @@ -0,0 +1,81 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import { normalizeCatalogModels, object } from "./catalog-normalization.ts"; +import { PROVIDER_CATALOG_OVERLAYS } from "./catalog-overlays.ts"; +import type { ModelProvider } from "./provider.ts"; +import { readBoundedJson, safeFetch } from "./transport-safety.ts"; + +/** Live facts use the same reviewed policy as the offline catalog generator. */ +export function enableStaticCatalogRefresh( + provider: ModelProvider, + fetchImpl?: typeof fetch, +): void { + const overlay = PROVIDER_CATALOG_OVERLAYS.find((entry) => entry.id === provider.id); + if (overlay?.source?.manifest !== "models-dev") return; + const sourceProviderId = overlay.source.providerId; + const streamModel = provider.streamModel?.bind(provider); + if (streamModel === undefined) throw new Error(`${provider.id} cannot dispatch refreshed models`); + provider.streamModel = (model, request) => { + const dialect = + overlay.dialectRules?.find((rule) => model.modelId.startsWith(rule.prefix))?.dialect ?? + overlay.dialect; + if ( + JSON.stringify(model.endpoint) !== JSON.stringify(overlay.endpoint) || + model.apiDialect !== dialect + ) { + throw new Error( + `${provider.id} cached model does not match reviewed endpoint and dialect policy`, + ); + } + return streamModel(model, request); + }; + provider.refreshModelCatalog = async (context) => { + const signal = AbortSignal.any([context.signal, AbortSignal.timeout(15_000)]); + const response = await safeFetch( + "https://models.dev/api.json", + { signal }, + { + label: "Model metadata source", + expectedOrigin: "https://models.dev", + ...(fetchImpl === undefined ? {} : { fetch: fetchImpl }), + }, + ); + if (!response.ok) { + await response.body?.cancel(); + throw new Error(`Model metadata source returned HTTP ${response.status}`); + } + const catalog = object(await readBoundedJson(response, 32 * 1024 * 1024, signal), "catalog"); + const source = object(catalog[sourceProviderId], "source provider"); + const models = Object.fromEntries( + Object.entries(object(source.models, "models")).map(([id, value]) => { + const model = object(value, "model"); + const limits = object(model.limit, "model limits"); + const modalities = object(model.modalities, "model modalities"); + return [ + id, + { + id: model.id, + name: model.name, + toolCall: model.tool_call, + structuredOutput: model.structured_output, + imageInput: Array.isArray(modalities.input) && modalities.input.includes("image"), + reasoning: model.reasoning, + reasoningOptions: model.reasoning_options, + contextWindow: limits.context, + maxOutputTokens: limits.output, + cost: model.cost, + status: model.status, + }, + ]; + }), + ); + return { + status: "updated", + providerId: provider.id, + generation: context.generation, + source: { id: "models.dev", kind: "provider_api" }, + models: normalizeCatalogModels(overlay, models), + }; + }; +} diff --git a/packages/ai/src/catalog-store.ts b/packages/ai/src/catalog-store.ts new file mode 100644 index 0000000..a25c17e --- /dev/null +++ b/packages/ai/src/catalog-store.ts @@ -0,0 +1,450 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { randomUUID } from "node:crypto"; +import { mkdir, open, readFile, rename, rm } from "node:fs/promises"; +import { resolve } from "node:path"; + +import { validateEndpointPolicy, validateModelCatalog } from "./catalog-validation.ts"; +import type { ImageModelInfo, ModelInfo } from "./model.ts"; + +const PROVIDER_IDENTIFIER = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; +const SOURCE_IDENTIFIER = /^[a-z0-9]+(?:[a-z0-9._:/-]*[a-z0-9])?$/i; +const MAX_SOURCE_TEXT_LENGTH = 512; +const MAX_ETAG_LENGTH = 1_024; +const LOCK_RETRY_MS = 25; +const LOCK_STALE_MS = 10_000; +const LOCK_TIMEOUT_MS = 5_000; + +export interface CatalogSourceMetadata { + /** Stable public source identity. This is not a URL or provider configuration object. */ + readonly id: string; + readonly kind: "provider_api" | "entitlement" | "gateway"; + /** Optional public source revision, never a credential or authorization value. */ + readonly revision?: string; +} + +/** Complete provider-scoped last-known-good catalog generation. */ +export interface CatalogSnapshot { + readonly version: 1; + readonly providerId: string; + readonly generation: number; + /** Time the source was last checked, as epoch milliseconds. */ + readonly checkedAt: number; + /** Time this model generation was accepted, as epoch milliseconds. */ + readonly updatedAt: number; + /** Source-provided model-data timestamp, as epoch milliseconds. */ + readonly sourceUpdatedAt?: number; + /** Opaque HTTP entity validator, when the source supports one. */ + readonly etag?: string; + readonly source: CatalogSourceMetadata; + readonly models: readonly ModelInfo[]; + readonly imageModels?: readonly ImageModelInfo[]; +} + +export interface CatalogStoreOperationOptions { + readonly signal?: AbortSignal; +} + +/** Provider-scoped last-known-good catalog persistence. */ +export interface CatalogStore { + read( + providerId: string, + options?: CatalogStoreOperationOptions, + ): Promise; + write( + providerId: string, + snapshot: CatalogSnapshot, + options?: CatalogStoreOperationOptions, + ): Promise; + delete(providerId: string, options?: CatalogStoreOperationOptions): Promise; +} + +export class CatalogStoreError extends Error { + readonly providerId: string; + + constructor(providerId: string, message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "CatalogStoreError"; + this.providerId = providerId; + } +} + +function fail(providerId: string, message: string): never { + throw new CatalogStoreError(providerId, `Catalog snapshot for ${providerId} ${message}`); +} + +function record(value: unknown, providerId: string): Record { + if ( + typeof value !== "object" || + value === null || + Array.isArray(value) || + Object.getPrototypeOf(value) !== Object.prototype + ) { + return fail(providerId, "must be an object"); + } + return value as Record; +} + +function exactKeys( + value: Record, + allowed: ReadonlySet, + providerId: string, + label: string, +): void { + for (const key of Object.keys(value)) { + if (!allowed.has(key)) fail(providerId, `${label} has unknown field ${JSON.stringify(key)}`); + } +} + +function validTimestamp(value: unknown): value is number { + return Number.isSafeInteger(value) && (value as number) >= 0; +} + +function hasControlCharacter(value: string): boolean { + return [...value].some((character) => { + const code = character.codePointAt(0) ?? 0; + return code < 32 || code === 127; + }); +} + +export function validateCatalogSource(value: unknown, providerId: string): CatalogSourceMetadata { + const source = record(value, providerId); + exactKeys(source, new Set(["id", "kind", "revision"]), providerId, "source metadata"); + if ( + typeof source.id !== "string" || + source.id.length > MAX_SOURCE_TEXT_LENGTH || + !SOURCE_IDENTIFIER.test(source.id) + ) { + fail(providerId, "has an invalid source identity"); + } + if ( + source.kind !== "provider_api" && + source.kind !== "entitlement" && + source.kind !== "gateway" + ) { + fail(providerId, "has an invalid source kind"); + } + if ( + source.revision !== undefined && + (typeof source.revision !== "string" || + source.revision.length === 0 || + source.revision.length > MAX_SOURCE_TEXT_LENGTH || + hasControlCharacter(source.revision)) + ) { + fail(providerId, "has an invalid source revision"); + } + return source as unknown as CatalogSourceMetadata; +} + +export function validateCatalogSnapshot( + value: unknown, + expectedProviderId: string, +): CatalogSnapshot { + if (!PROVIDER_IDENTIFIER.test(expectedProviderId)) { + throw new CatalogStoreError( + expectedProviderId, + `Invalid catalog provider ID ${expectedProviderId}`, + ); + } + const snapshot = record(value, expectedProviderId); + exactKeys( + snapshot, + new Set([ + "version", + "providerId", + "generation", + "checkedAt", + "updatedAt", + "sourceUpdatedAt", + "etag", + "source", + "models", + "imageModels", + ]), + expectedProviderId, + "snapshot", + ); + if (snapshot.version !== 1) fail(expectedProviderId, "has an unsupported version"); + if (snapshot.providerId !== expectedProviderId) + fail(expectedProviderId, "has a mismatched provider ID"); + if (!Number.isSafeInteger(snapshot.generation) || (snapshot.generation as number) < 1) { + fail(expectedProviderId, "has an invalid generation"); + } + if (!validTimestamp(snapshot.checkedAt) || !validTimestamp(snapshot.updatedAt)) { + fail(expectedProviderId, "has invalid freshness timestamps"); + } + if (snapshot.sourceUpdatedAt !== undefined && !validTimestamp(snapshot.sourceUpdatedAt)) { + fail(expectedProviderId, "has an invalid source timestamp"); + } + if ( + snapshot.etag !== undefined && + (typeof snapshot.etag !== "string" || + snapshot.etag.length === 0 || + snapshot.etag.length > MAX_ETAG_LENGTH || + /[\r\n]/.test(snapshot.etag)) + ) { + fail(expectedProviderId, "has an invalid ETag"); + } + validateCatalogSource(snapshot.source, expectedProviderId); + if (!Array.isArray(snapshot.models)) fail(expectedProviderId, "has a non-array model catalog"); + try { + validateModelCatalog(snapshot.models as readonly ModelInfo[]); + } catch (error) { + throw new CatalogStoreError( + expectedProviderId, + `Catalog snapshot for ${expectedProviderId} is invalid`, + { + cause: error, + }, + ); + } + for (const model of snapshot.models as readonly ModelInfo[]) { + if (model.providerId !== expectedProviderId) { + fail(expectedProviderId, `contains model ${model.modelId} owned by ${model.providerId}`); + } + } + if (snapshot.imageModels !== undefined) { + if (!Array.isArray(snapshot.imageModels)) + fail(expectedProviderId, "has a non-array image catalog"); + const imageIds = new Set(); + for (const model of snapshot.imageModels as readonly ImageModelInfo[]) { + if ( + typeof model !== "object" || + model === null || + model.providerId !== expectedProviderId || + typeof model.modelId !== "string" || + model.modelId.length === 0 || + typeof model.displayName !== "string" || + model.displayName.length === 0 || + model.apiDialect !== "openrouter-images" || + !Array.isArray(model.input) || + !model.input.includes("text") || + !Array.isArray(model.output) || + !model.output.includes("image") || + imageIds.has(model.modelId) + ) { + fail(expectedProviderId, "contains an invalid image model"); + } + if (model.endpoint !== undefined) validateEndpointPolicy(model.endpoint, model.modelId); + imageIds.add(model.modelId); + } + } + return structuredClone(snapshot) as unknown as CatalogSnapshot; +} + +export class InMemoryCatalogStore implements CatalogStore { + private readonly snapshots = new Map(); + + async read( + providerId: string, + options: CatalogStoreOperationOptions = {}, + ): Promise { + options.signal?.throwIfAborted(); + const snapshot = this.snapshots.get(providerId); + return snapshot === undefined ? undefined : structuredClone(snapshot); + } + + async write( + providerId: string, + snapshot: CatalogSnapshot, + options: CatalogStoreOperationOptions = {}, + ): Promise { + options.signal?.throwIfAborted(); + this.snapshots.set(providerId, validateCatalogSnapshot(snapshot, providerId)); + } + + async delete(providerId: string, options: CatalogStoreOperationOptions = {}): Promise { + options.signal?.throwIfAborted(); + this.snapshots.delete(providerId); + } +} + +function sleep(ms: number, signal?: AbortSignal): Promise { + signal?.throwIfAborted(); + return new Promise((resolvePromise, reject) => { + const finish = () => { + signal?.removeEventListener("abort", abort); + resolvePromise(); + }; + const timer = setTimeout(finish, ms); + const abort = () => { + clearTimeout(timer); + signal?.removeEventListener("abort", abort); + reject(signal?.reason); + }; + signal?.addEventListener("abort", abort, { once: true }); + }); +} + +/** Atomic JSON store using one independently locked file per provider. */ +export class FileCatalogStore implements CatalogStore { + readonly directory: string; + private readonly tails = new Map>(); + + constructor(directory: string) { + this.directory = resolve(directory); + } + + read( + providerId: string, + options: CatalogStoreOperationOptions = {}, + ): Promise { + return this.enqueue(providerId, async () => { + options.signal?.throwIfAborted(); + const path = this.snapshotPath(providerId); + let raw: string; + try { + raw = await readFile(path, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return undefined; + throw new CatalogStoreError(providerId, `Cannot read catalog snapshot ${path}`, { + cause: error, + }); + } + options.signal?.throwIfAborted(); + let parsed: unknown; + try { + parsed = JSON.parse(raw); + } catch (error) { + throw new CatalogStoreError(providerId, `Catalog snapshot ${path} is not valid JSON`, { + cause: error, + }); + } + return validateCatalogSnapshot(parsed, providerId); + }); + } + + write( + providerId: string, + snapshot: CatalogSnapshot, + options: CatalogStoreOperationOptions = {}, + ): Promise { + return this.enqueue(providerId, () => + this.withLock(providerId, options.signal, async () => { + options.signal?.throwIfAborted(); + const validated = validateCatalogSnapshot(snapshot, providerId); + await mkdir(this.directory, { recursive: true, mode: 0o700 }); + const path = this.snapshotPath(providerId); + const temporary = `${path}.${process.pid}.${randomUUID()}.tmp`; + try { + const handle = await open(temporary, "wx", 0o600); + try { + await handle.writeFile(`${JSON.stringify(validated, null, "\t")}\n`); + await handle.sync(); + } finally { + await handle.close(); + } + options.signal?.throwIfAborted(); + await rename(temporary, path); + } catch (error) { + options.signal?.throwIfAborted(); + throw new CatalogStoreError(providerId, `Cannot write catalog snapshot ${path}`, { + cause: error, + }); + } finally { + await rm(temporary, { force: true }); + } + }), + ); + } + + delete(providerId: string, options: CatalogStoreOperationOptions = {}): Promise { + return this.enqueue(providerId, () => + this.withLock(providerId, options.signal, async () => { + options.signal?.throwIfAborted(); + try { + await rm(this.snapshotPath(providerId), { force: true }); + } catch (error) { + throw new CatalogStoreError(providerId, "Cannot delete catalog snapshot", { + cause: error, + }); + } + }), + ); + } + + private snapshotPath(providerId: string): string { + if (!PROVIDER_IDENTIFIER.test(providerId)) { + throw new CatalogStoreError(providerId, `Invalid catalog provider ID ${providerId}`); + } + return resolve(this.directory, `${providerId}.json`); + } + + private enqueue(providerId: string, task: () => Promise): Promise { + const previous = this.tails.get(providerId) ?? Promise.resolve(); + const queued = previous.then(task, task); + const tail = queued.then( + () => undefined, + () => undefined, + ); + this.tails.set(providerId, tail); + void tail.then(() => { + if (this.tails.get(providerId) === tail) this.tails.delete(providerId); + }); + return queued; + } + + private async withLock( + providerId: string, + signal: AbortSignal | undefined, + task: () => Promise, + ): Promise { + await mkdir(this.directory, { recursive: true, mode: 0o700 }); + const lockPath = `${this.snapshotPath(providerId)}.lock`; + const deadline = Date.now() + LOCK_TIMEOUT_MS; + for (;;) { + signal?.throwIfAborted(); + try { + const handle = await open(lockPath, "wx", 0o600); + try { + await handle.writeFile(`${process.pid} ${Date.now()}\n`); + } finally { + await handle.close(); + } + break; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "EEXIST") { + throw new CatalogStoreError(providerId, "Cannot lock catalog snapshot", { cause: error }); + } + if (await this.removeStaleLock(lockPath)) continue; + if (Date.now() >= deadline) { + throw new CatalogStoreError(providerId, "Timed out locking catalog snapshot"); + } + await sleep(LOCK_RETRY_MS, signal); + } + } + try { + signal?.throwIfAborted(); + return await task(); + } finally { + await rm(lockPath, { force: true }); + } + } + + private async removeStaleLock(lockPath: string): Promise { + try { + const contents = await readFile(lockPath, "utf8"); + const [rawPid, rawTime] = contents.trim().split(" "); + const pid = Number(rawPid); + const lockedAt = Number(rawTime); + if ( + !Number.isSafeInteger(pid) || + !Number.isFinite(lockedAt) || + Date.now() - lockedAt <= LOCK_STALE_MS + ) { + return false; + } + try { + process.kill(pid, 0); + return false; + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ESRCH") return false; + } + await rm(lockPath, { force: true }); + return true; + } catch { + return false; + } + } +} diff --git a/packages/ai/src/catalog-validation.ts b/packages/ai/src/catalog-validation.ts new file mode 100644 index 0000000..40caadb --- /dev/null +++ b/packages/ai/src/catalog-validation.ts @@ -0,0 +1,436 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { supportedThinkingLevels } from "@axl/protocol"; +import type { + EndpointPolicy, + ModelCachePolicy, + ModelCompatibility, + ModelInfo, + ModelSamplingPolicy, +} from "./model.ts"; +import { safeEndpoint } from "./transport-safety.ts"; + +const IDENTIFIER = /^[a-z0-9@](?:[a-z0-9._:/@-]*[a-z0-9])?$/i; +const PROVIDER_IDENTIFIER = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; +const SETTING_IDENTIFIER = /^[A-Za-z][A-Za-z0-9]*$/; +const SECRET_NAME = /(?:authorization|api[-_]?key|credential|password|secret|token)/i; +const FORBIDDEN_HEADER = /^(?:authorization|cookie|proxy-authorization|set-cookie|x-api-key)$/i; +const THINKING_LEVELS = new Set(["off", "minimal", "low", "medium", "high", "xhigh", "max"]); +const API_DIALECTS = new Set([ + "openai-chat", + "openai-responses", + "azure-openai-responses", + "openai-codex-responses", + "anthropic-messages", + "google-generative-ai", + "google-vertex", + "bedrock-converse-stream", + "mistral-conversations", + "gateway-messages", + "fake", +]); +const AVAILABILITY_STATUSES = new Set(["available", "preview", "deprecated", "unavailable"]); +const RETENTIONS = new Set(["none", "short", "long"]); +const SAMPLING_OPTIONS = new Set([ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", +]); +const MODEL_FIELDS = new Set([ + "providerId", + "modelId", + "displayName", + "apiDialect", + "capabilities", + "reasoning", + "thinkingLevelMap", + "contextWindow", + "maxOutputTokens", + "cost", + "cache", + "sampling", + "endpoint", + "availability", + "headers", + "compatibility", +]); +const CAPABILITY_FIELDS = new Set(["toolUse", "structuredOutput", "imageInput"]); +const COST_FIELDS = new Set([ + "inputUsdPerMTok", + "outputUsdPerMTok", + "cacheReadUsdPerMTok", + "cacheWriteUsdPerMTok", + "tiers", +]); +const COST_TIER_FIELDS = new Set( + [...COST_FIELDS].filter((field) => field !== "tiers").concat("inputTokensAbove"), +); +const CACHE_FIELDS = new Set(["supported", "defaultRetention", "supportedRetentions"]); +const SAMPLING_FIELDS = new Set(["supported", "customFields"]); +const AVAILABILITY_FIELDS = new Set(["status", "reason"]); +const ENDPOINT_FIELDS = { + fixed: new Set(["type", "baseUrl"]), + configured: new Set(["type", "baseUrlSetting", "defaultBaseUrl"]), + template: new Set(["type", "template", "variables"]), +} as const; +const ENDPOINT_VARIABLE_FIELDS = new Set(["name", "setting", "required", "defaultValue"]); +const COMPATIBILITY_FIELDS = new Set([ + "dialect", + "supportsStore", + "supportsDeveloperRole", + "supportsReasoningEffort", + "supportsUsageInStreaming", + "supportsFinishReason", + "maxTokensField", + "requiresToolResultName", + "requiresAssistantAfterToolResult", + "requiresThinkingAsText", + "requiresReasoningContentOnAssistantMessages", + "thinkingFormat", + "thinkingTokenBudgetField", + "supportsGrammarTools", + "supportsStrictTools", + "cacheControlFormat", + "sessionAffinityFormat", + "supportsLongCacheRetention", + "supportsMaxOutputTokens", + "routing", + "supportsCacheControlOnTools", + "supportsTemperature", + "forceAdaptiveThinking", + "allowEmptyThinkingSignature", + "supportsPromptCacheMarkers", + "supportsThinkingSignatures", +]); +const ROUTING_FIELDS = new Set([ + "only", + "order", + "ignore", + "allowFallbacks", + "requireParameters", + "dataCollection", + "zeroDataRetention", + "sort", +]); + +function rejectUnknownFields( + value: object, + allowed: ReadonlySet, + label: string, + errors: string[], +): void { + for (const key of Object.keys(value)) { + if (!allowed.has(key)) errors.push(`${label} has unknown field ${JSON.stringify(key)}`); + } +} + +export interface BuiltinCatalogProvider { + readonly id: string; + readonly displayName: string; + readonly catalogKind: "static" | "dynamic" | "configured"; + readonly regionFamily?: string; + readonly region?: string; +} + +export interface CatalogProvenance { + readonly generatedAt: string; + readonly sources: readonly { + readonly name: string; + readonly location: string; + readonly retrievedAt: string; + readonly sha256?: string; + readonly revision?: string; + readonly license: string; + }[]; +} + +export class ModelCatalogValidationError extends Error { + readonly errors: readonly string[]; + + constructor(errors: readonly string[]) { + super(`Invalid model catalog:\n${errors.map((error) => ` * ${error}`).join("\n")}`); + this.name = "ModelCatalogValidationError"; + this.errors = errors; + } +} + +function validUrl(value: string, label: string, errors: string[]): void { + try { + safeEndpoint(value, { label, allowLoopbackHttp: true }); + } catch (error) { + errors.push(error instanceof Error ? error.message : `${label} is not a valid URL`); + } +} + +function collectEndpointErrors(endpoint: EndpointPolicy, label: string, errors: string[]): void { + rejectUnknownFields(endpoint, ENDPOINT_FIELDS[endpoint.type], `${label} endpoint`, errors); + if (endpoint.type === "fixed") { + validUrl(endpoint.baseUrl, `${label} endpoint`, errors); + return; + } + if (endpoint.type === "configured") { + if (!SETTING_IDENTIFIER.test(endpoint.baseUrlSetting)) { + errors.push(`${label} has an invalid endpoint setting`); + } + if (SECRET_NAME.test(endpoint.baseUrlSetting)) { + errors.push(`${label} endpoint setting must not name a credential`); + } + if (endpoint.defaultBaseUrl !== undefined) { + validUrl(endpoint.defaultBaseUrl, `${label} default endpoint`, errors); + } + return; + } + + const names = new Set(); + for (const variable of endpoint.variables) { + rejectUnknownFields(variable, ENDPOINT_VARIABLE_FIELDS, `${label} endpoint variable`, errors); + if (!PROVIDER_IDENTIFIER.test(variable.name) || names.has(variable.name)) { + errors.push(`${label} has an invalid or duplicate endpoint variable ${variable.name}`); + } + names.add(variable.name); + if (!SETTING_IDENTIFIER.test(variable.setting) || SECRET_NAME.test(variable.setting)) { + errors.push(`${label} endpoint variable ${variable.name} has an unsafe setting`); + } + const occurrences = endpoint.template.split(`{${variable.name}}`).length - 1; + if (occurrences === 0) errors.push(`${label} endpoint does not use variable ${variable.name}`); + if (!variable.required && variable.defaultValue === undefined) { + errors.push(`${label} optional endpoint variable ${variable.name} needs a default`); + } + } + const placeholders = [...endpoint.template.matchAll(/\{([^{}]+)\}/g)].map((match) => match[1]); + if (placeholders.some((name) => name === undefined || !names.has(name))) { + errors.push(`${label} endpoint contains an undeclared variable`); + } + validUrl( + endpoint.template.replaceAll(/\{[^{}]+\}/g, "catalog-value"), + `${label} endpoint template`, + errors, + ); +} + +export function validateEndpointPolicy(endpoint: EndpointPolicy, label = "catalog"): void { + const errors: string[] = []; + collectEndpointErrors(endpoint, label, errors); + if (errors.length > 0) throw new ModelCatalogValidationError(errors); +} + +function validateCost(model: ModelInfo, label: string, errors: string[]): void { + if (model.cost === undefined) return; + rejectUnknownFields(model.cost, COST_FIELDS, `${label} cost`, errors); + const rates = [ + model.cost.inputUsdPerMTok, + model.cost.outputUsdPerMTok, + model.cost.cacheReadUsdPerMTok, + model.cost.cacheWriteUsdPerMTok, + ]; + if (rates.some((rate) => rate !== undefined && (!Number.isFinite(rate) || rate < 0))) { + errors.push(`${label} has invalid pricing`); + } + let threshold = -1; + for (const tier of model.cost.tiers ?? []) { + rejectUnknownFields(tier, COST_TIER_FIELDS, `${label} cost tier`, errors); + if (!Number.isSafeInteger(tier.inputTokensAbove) || tier.inputTokensAbove <= threshold) { + errors.push(`${label} has unsorted or invalid price tiers`); + } + threshold = tier.inputTokensAbove; + if ( + [ + tier.inputUsdPerMTok, + tier.outputUsdPerMTok, + tier.cacheReadUsdPerMTok, + tier.cacheWriteUsdPerMTok, + ].some((rate) => rate !== undefined && (!Number.isFinite(rate) || rate < 0)) + ) { + errors.push(`${label} has invalid tier pricing`); + } + } +} + +function validateCache(cache: ModelCachePolicy, label: string, errors: string[]): void { + rejectUnknownFields(cache, CACHE_FIELDS, `${label} cache`, errors); + if (!RETENTIONS.has(cache.defaultRetention) || cache.supportedRetentions.length === 0) { + errors.push(`${label} has invalid cache retention metadata`); + return; + } + const retentions = new Set(cache.supportedRetentions); + if ( + retentions.size !== cache.supportedRetentions.length || + !retentions.has(cache.defaultRetention) + ) { + errors.push(`${label} has inconsistent cache retentions`); + } + if (cache.supported && !retentions.has("none")) { + errors.push(`${label} cache policy must allow no retention`); + } + if (!cache.supported && (cache.defaultRetention !== "none" || retentions.size !== 1)) { + errors.push(`${label} unsupported cache policy must contain only none`); + } +} + +function validateSampling(sampling: ModelSamplingPolicy, label: string, errors: string[]): void { + if (typeof sampling !== "object" || sampling === null || Array.isArray(sampling)) { + errors.push(`${label} has invalid sampling support`); + return; + } + rejectUnknownFields(sampling, SAMPLING_FIELDS, `${label} sampling`, errors); + if (!Array.isArray(sampling.supported)) { + errors.push(`${label} has invalid sampling support`); + return; + } + const supported = new Set(sampling.supported); + if ( + supported.size !== sampling.supported.length || + sampling.supported.some((option) => !SAMPLING_OPTIONS.has(option)) + ) { + errors.push(`${label} has invalid sampling support`); + } + if ( + sampling.customFields !== undefined && + (!Array.isArray(sampling.customFields) || + new Set(sampling.customFields).size !== sampling.customFields.length || + sampling.customFields.some( + (field) => + typeof field !== "string" || field.trim().length === 0 || forbiddenMetadataName(field), + )) + ) { + errors.push(`${label} has invalid custom sampling fields`); + } +} + +function forbiddenMetadataName(name: string): boolean { + const normalized = name.replace(/[^a-z0-9]/gi, "").toLowerCase(); + return ["authorization", "cookie", "credential", "password", "secret", "token", "apikey"].some( + (forbidden) => normalized === forbidden || normalized.endsWith(forbidden), + ); +} + +function validateCompatibility( + compatibility: ModelCompatibility, + model: ModelInfo, + label: string, + errors: string[], +): void { + rejectUnknownFields(compatibility, COMPATIBILITY_FIELDS, `${label} compatibility`, errors); + if (compatibility.dialect !== model.apiDialect) { + errors.push(`${label} compatibility dialect does not match its API dialect`); + } + if ("routing" in compatibility && compatibility.routing !== undefined) { + rejectUnknownFields(compatibility.routing, ROUTING_FIELDS, `${label} routing`, errors); + } +} + +export function validateModelCatalog(models: readonly ModelInfo[]): readonly ModelInfo[] { + const errors: string[] = []; + const identities = new Set(); + for (const model of models) { + const label = `${model.providerId}/${model.modelId}`; + rejectUnknownFields(model, MODEL_FIELDS, label, errors); + rejectUnknownFields(model.capabilities, CAPABILITY_FIELDS, `${label} capabilities`, errors); + if (!PROVIDER_IDENTIFIER.test(model.providerId)) + errors.push(`${label} has an invalid provider ID`); + if (!IDENTIFIER.test(model.modelId) || model.modelId.length > 256) { + errors.push(`${label} has an invalid model ID`); + } + if (identities.has(label)) errors.push(`${label} is duplicated`); + identities.add(label); + if (model.displayName.trim().length === 0 || model.displayName !== model.displayName.trim()) { + errors.push(`${label} has an invalid display name`); + } + const legacyCompatibility = + model.compatibility !== undefined && + !("dialect" in model.compatibility && typeof model.compatibility.dialect === "string"); + if (!API_DIALECTS.has(model.apiDialect) && !legacyCompatibility) + errors.push(`${label} has an invalid API dialect`); + if ( + typeof model.capabilities.toolUse !== "boolean" || + typeof model.capabilities.structuredOutput !== "boolean" || + typeof model.capabilities.imageInput !== "boolean" + ) { + errors.push(`${label} has invalid capabilities`); + } + if (typeof model.reasoning !== "boolean") + errors.push(`${label} has invalid reasoning metadata`); + if (!Number.isSafeInteger(model.contextWindow) || model.contextWindow <= 0) { + errors.push(`${label} has an invalid context window`); + } + if ( + !Number.isSafeInteger(model.maxOutputTokens) || + model.maxOutputTokens <= 0 || + model.maxOutputTokens > model.contextWindow + ) { + errors.push(`${label} has an invalid output limit`); + } + if (!model.reasoning && model.thinkingLevelMap !== undefined) { + errors.push(`${label} has a reasoning map but no reasoning capability`); + } + if (model.thinkingLevelMap !== undefined) { + const map = model.thinkingLevelMap; + if (typeof map !== "object" || map === null || Array.isArray(map)) { + errors.push(`${label} has an invalid reasoning map`); + } else { + for (const [level, value] of Object.entries(map)) { + if ( + !THINKING_LEVELS.has(level) || + (value !== null && (typeof value !== "string" || value.trim().length === 0)) + ) { + errors.push(`${label} has an invalid reasoning map`); + } + } + if (supportedThinkingLevels(model).length === 0) { + errors.push(`${label} has no supported thinking level`); + } + } + } + validateCost(model, label, errors); + if (model.cache !== undefined) { + validateCache(model.cache, label, errors); + if ( + !model.cache.supported && + ((model.cost?.cacheReadUsdPerMTok ?? 0) > 0 || (model.cost?.cacheWriteUsdPerMTok ?? 0) > 0) + ) { + errors.push(`${label} has cache pricing but caching is unsupported`); + } + } + if (model.sampling !== undefined) validateSampling(model.sampling, label, errors); + if (model.availability !== undefined) { + rejectUnknownFields(model.availability, AVAILABILITY_FIELDS, `${label} availability`, errors); + } + if ( + model.availability !== undefined && + (!AVAILABILITY_STATUSES.has(model.availability.status) || + (model.availability.reason !== undefined && model.availability.reason.trim().length === 0)) + ) { + errors.push(`${label} has invalid availability metadata`); + } + if (model.endpoint !== undefined) collectEndpointErrors(model.endpoint, label, errors); + if (model.compatibility !== undefined) { + if ("dialect" in model.compatibility && typeof model.compatibility.dialect === "string") { + validateCompatibility(model.compatibility as ModelCompatibility, model, label, errors); + } else if ( + Object.entries(model.compatibility).some( + ([name, value]) => forbiddenMetadataName(name) || typeof value !== "boolean", + ) + ) { + errors.push(`${label} has invalid legacy compatibility metadata`); + } + } + for (const [name, value] of Object.entries(model.headers ?? {})) { + if ( + FORBIDDEN_HEADER.test(name) || + forbiddenMetadataName(name) || + /[\r\n]/.test(name) || + /[\r\n]/.test(value) + ) { + errors.push(`${label} contains an unsafe static header`); + } + } + } + if (errors.length > 0) throw new ModelCatalogValidationError(errors); + return models; +} diff --git a/packages/ai/src/catalog.generated.ts b/packages/ai/src/catalog.generated.ts new file mode 100644 index 0000000..4cf3de3 --- /dev/null +++ b/packages/ai/src/catalog.generated.ts @@ -0,0 +1,128 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { BuiltinCatalogProvider, CatalogProvenance } from "./catalog.ts"; +import type { ModelInfo } from "./model.ts"; +import { MODELS as AMAZON_BEDROCK_MODELS } from "./catalog.generated/amazon-bedrock.generated.ts"; +import { MODELS as ANT_LING_MODELS } from "./catalog.generated/ant-ling.generated.ts"; +import { MODELS as ANTHROPIC_MODELS } from "./catalog.generated/anthropic.generated.ts"; +import { MODELS as AZURE_OPENAI_RESPONSES_MODELS } from "./catalog.generated/azure-openai-responses.generated.ts"; +import { MODELS as BASETEN_MODELS } from "./catalog.generated/baseten.generated.ts"; +import { MODELS as CEREBRAS_MODELS } from "./catalog.generated/cerebras.generated.ts"; +import { MODELS as CLOUDFLARE_WORKERS_AI_MODELS } from "./catalog.generated/cloudflare-workers-ai.generated.ts"; +import { MODELS as DEEPSEEK_MODELS } from "./catalog.generated/deepseek.generated.ts"; +import { MODELS as FIREWORKS_MODELS } from "./catalog.generated/fireworks.generated.ts"; +import { MODELS as GOOGLE_MODELS } from "./catalog.generated/google.generated.ts"; +import { MODELS as GOOGLE_VERTEX_MODELS } from "./catalog.generated/google-vertex.generated.ts"; +import { MODELS as GROQ_MODELS } from "./catalog.generated/groq.generated.ts"; +import { MODELS as HUGGINGFACE_MODELS } from "./catalog.generated/huggingface.generated.ts"; +import { MODELS as KIMI_CODING_MODELS } from "./catalog.generated/kimi-coding.generated.ts"; +import { MODELS as MINIMAX_MODELS } from "./catalog.generated/minimax.generated.ts"; +import { MODELS as MINIMAX_CN_MODELS } from "./catalog.generated/minimax-cn.generated.ts"; +import { MODELS as MISTRAL_MODELS } from "./catalog.generated/mistral.generated.ts"; +import { MODELS as MOONSHOTAI_MODELS } from "./catalog.generated/moonshotai.generated.ts"; +import { MODELS as MOONSHOTAI_CN_MODELS } from "./catalog.generated/moonshotai-cn.generated.ts"; +import { MODELS as NVIDIA_MODELS } from "./catalog.generated/nvidia.generated.ts"; +import { MODELS as OPENAI_MODELS } from "./catalog.generated/openai.generated.ts"; +import { MODELS as OPENAI_CODEX_MODELS } from "./catalog.generated/openai-codex.generated.ts"; +import { MODELS as OPENCODE_MODELS } from "./catalog.generated/opencode.generated.ts"; +import { MODELS as OPENCODE_GO_MODELS } from "./catalog.generated/opencode-go.generated.ts"; +import { MODELS as QWEN_TOKEN_PLAN_MODELS } from "./catalog.generated/qwen-token-plan.generated.ts"; +import { MODELS as QWEN_TOKEN_PLAN_CN_MODELS } from "./catalog.generated/qwen-token-plan-cn.generated.ts"; +import { MODELS as QWEN_TOKEN_PLAN_INDIVIDUAL_MODELS } from "./catalog.generated/qwen-token-plan-individual.generated.ts"; +import { MODELS as TOGETHER_MODELS } from "./catalog.generated/together.generated.ts"; +import { MODELS as VERCEL_AI_GATEWAY_MODELS } from "./catalog.generated/vercel-ai-gateway.generated.ts"; +import { MODELS as XAI_MODELS } from "./catalog.generated/xai.generated.ts"; +import { MODELS as XIAOMI_MODELS } from "./catalog.generated/xiaomi.generated.ts"; +import { MODELS as XIAOMI_TOKEN_PLAN_AMS_MODELS } from "./catalog.generated/xiaomi-token-plan-ams.generated.ts"; +import { MODELS as XIAOMI_TOKEN_PLAN_CN_MODELS } from "./catalog.generated/xiaomi-token-plan-cn.generated.ts"; +import { MODELS as XIAOMI_TOKEN_PLAN_SGP_MODELS } from "./catalog.generated/xiaomi-token-plan-sgp.generated.ts"; +import { MODELS as ZAI_MODELS } from "./catalog.generated/zai.generated.ts"; +import { MODELS as ZAI_CODING_CN_MODELS } from "./catalog.generated/zai-coding-cn.generated.ts"; + +export const GENERATED_CATALOG_PROVENANCE = {"generatedAt":"2026-09-05T13:49:08Z","sources":[{"name":"models.dev","location":"https://models.dev/api.json","retrievedAt":"2026-09-05T13:49:08Z","sha256":"0b09a4d8dedab6a804ca15046729bb2ec03c5f5b689b89a983ea488bb71eaeef","revision":"5c600a037417cf778ee6eb3ea2ce0f17abc12130","license":"MIT"},{"name":"Ant Ling official documentation","location":"https://developer.ant-ling.com/en/docs/api-reference/","retrievedAt":"2026-09-05T13:49:08Z","sha256":"3c715bab4b49c6b76d5f65ba7d3545ff20e9ed1fb793e9183cbf488ce47dd1ae","license":"factual metadata"}]} as const satisfies CatalogProvenance; + +export const BUILTIN_CATALOG_PROVIDERS = [ + {"id":"amazon-bedrock","displayName":"Amazon Bedrock","catalogKind":"static"}, + {"id":"ant-ling","displayName":"Ant Ling","catalogKind":"static"}, + {"id":"anthropic","displayName":"Anthropic","catalogKind":"static"}, + {"id":"azure-openai-responses","displayName":"Azure OpenAI Responses","catalogKind":"static"}, + {"id":"baseten","displayName":"Baseten","catalogKind":"static"}, + {"id":"cerebras","displayName":"Cerebras","catalogKind":"static"}, + {"id":"cloudflare-ai-gateway","displayName":"Cloudflare AI Gateway","catalogKind":"dynamic"}, + {"id":"cloudflare-workers-ai","displayName":"Cloudflare Workers AI","catalogKind":"static"}, + {"id":"custom","displayName":"User configured endpoint","catalogKind":"configured"}, + {"id":"deepseek","displayName":"DeepSeek","catalogKind":"static"}, + {"id":"fireworks","displayName":"Fireworks AI","catalogKind":"static"}, + {"id":"github-copilot","displayName":"GitHub Copilot","catalogKind":"dynamic"}, + {"id":"google","displayName":"Google Generative AI","catalogKind":"static"}, + {"id":"google-vertex","displayName":"Google Vertex AI","catalogKind":"static"}, + {"id":"groq","displayName":"Groq","catalogKind":"static"}, + {"id":"huggingface","displayName":"Hugging Face","catalogKind":"static"}, + {"id":"kimi-coding","displayName":"Kimi For Coding","catalogKind":"static"}, + {"id":"minimax","displayName":"MiniMax","catalogKind":"static","regionFamily":"minimax","region":"global"}, + {"id":"minimax-cn","displayName":"MiniMax China","catalogKind":"static","regionFamily":"minimax","region":"cn"}, + {"id":"mistral","displayName":"Mistral","catalogKind":"static"}, + {"id":"moonshotai","displayName":"Moonshot AI","catalogKind":"static","regionFamily":"moonshotai","region":"global"}, + {"id":"moonshotai-cn","displayName":"Moonshot AI China","catalogKind":"static","regionFamily":"moonshotai","region":"cn"}, + {"id":"nvidia","displayName":"NVIDIA NIM","catalogKind":"static"}, + {"id":"openai","displayName":"OpenAI","catalogKind":"static"}, + {"id":"openai-codex","displayName":"OpenAI Codex","catalogKind":"static"}, + {"id":"opencode","displayName":"OpenCode Zen","catalogKind":"static"}, + {"id":"opencode-go","displayName":"OpenCode Go","catalogKind":"static"}, + {"id":"openrouter","displayName":"OpenRouter","catalogKind":"dynamic"}, + {"id":"qwen-token-plan","displayName":"Qwen Token Plan","catalogKind":"static","regionFamily":"qwen-token-plan","region":"sgp"}, + {"id":"qwen-token-plan-cn","displayName":"Qwen Token Plan China","catalogKind":"static","regionFamily":"qwen-token-plan","region":"cn"}, + {"id":"qwen-token-plan-individual","displayName":"Qwen Token Plan Individual","catalogKind":"static"}, + {"id":"radius","displayName":"Radius","catalogKind":"dynamic"}, + {"id":"together","displayName":"Together AI","catalogKind":"static"}, + {"id":"vercel-ai-gateway","displayName":"Vercel AI Gateway","catalogKind":"static"}, + {"id":"xai","displayName":"xAI","catalogKind":"static"}, + {"id":"xiaomi","displayName":"Xiaomi MiMo","catalogKind":"static","regionFamily":"xiaomi","region":"global"}, + {"id":"xiaomi-token-plan-ams","displayName":"Xiaomi Token Plan Amsterdam","catalogKind":"static","regionFamily":"xiaomi","region":"ams"}, + {"id":"xiaomi-token-plan-cn","displayName":"Xiaomi Token Plan China","catalogKind":"static","regionFamily":"xiaomi","region":"cn"}, + {"id":"xiaomi-token-plan-sgp","displayName":"Xiaomi Token Plan Singapore","catalogKind":"static","regionFamily":"xiaomi","region":"sgp"}, + {"id":"zai","displayName":"Z.AI","catalogKind":"static","regionFamily":"zai","region":"global"}, + {"id":"zai-coding-cn","displayName":"Z.AI Coding China","catalogKind":"static","regionFamily":"zai","region":"cn"}, +] as const satisfies readonly BuiltinCatalogProvider[]; + +export const STATIC_MODEL_CATALOG: Readonly> = { + "amazon-bedrock": AMAZON_BEDROCK_MODELS, + "ant-ling": ANT_LING_MODELS, + "anthropic": ANTHROPIC_MODELS, + "azure-openai-responses": AZURE_OPENAI_RESPONSES_MODELS, + "baseten": BASETEN_MODELS, + "cerebras": CEREBRAS_MODELS, + "cloudflare-workers-ai": CLOUDFLARE_WORKERS_AI_MODELS, + "deepseek": DEEPSEEK_MODELS, + "fireworks": FIREWORKS_MODELS, + "google": GOOGLE_MODELS, + "google-vertex": GOOGLE_VERTEX_MODELS, + "groq": GROQ_MODELS, + "huggingface": HUGGINGFACE_MODELS, + "kimi-coding": KIMI_CODING_MODELS, + "minimax": MINIMAX_MODELS, + "minimax-cn": MINIMAX_CN_MODELS, + "mistral": MISTRAL_MODELS, + "moonshotai": MOONSHOTAI_MODELS, + "moonshotai-cn": MOONSHOTAI_CN_MODELS, + "nvidia": NVIDIA_MODELS, + "openai": OPENAI_MODELS, + "openai-codex": OPENAI_CODEX_MODELS, + "opencode": OPENCODE_MODELS, + "opencode-go": OPENCODE_GO_MODELS, + "qwen-token-plan": QWEN_TOKEN_PLAN_MODELS, + "qwen-token-plan-cn": QWEN_TOKEN_PLAN_CN_MODELS, + "qwen-token-plan-individual": QWEN_TOKEN_PLAN_INDIVIDUAL_MODELS, + "together": TOGETHER_MODELS, + "vercel-ai-gateway": VERCEL_AI_GATEWAY_MODELS, + "xai": XAI_MODELS, + "xiaomi": XIAOMI_MODELS, + "xiaomi-token-plan-ams": XIAOMI_TOKEN_PLAN_AMS_MODELS, + "xiaomi-token-plan-cn": XIAOMI_TOKEN_PLAN_CN_MODELS, + "xiaomi-token-plan-sgp": XIAOMI_TOKEN_PLAN_SGP_MODELS, + "zai": ZAI_MODELS, + "zai-coding-cn": ZAI_CODING_CN_MODELS, +}; diff --git a/packages/ai/src/catalog.generated/amazon-bedrock.generated.ts b/packages/ai/src/catalog.generated/amazon-bedrock.generated.ts new file mode 100644 index 0000000..b6c8851 --- /dev/null +++ b/packages/ai/src/catalog.generated/amazon-bedrock.generated.ts @@ -0,0 +1,131 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"amazon-bedrock","modelId":"amazon.nova-2-lite-v1:0","displayName":"Nova 2 Lite","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.33,"outputUsdPerMTok":2.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"amazon.nova-lite-v1:0","displayName":"Nova Lite","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.24,"cacheReadUsdPerMTok":0.015},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"amazon.nova-micro-v1:0","displayName":"Nova Micro","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.035,"outputUsdPerMTok":0.14,"cacheReadUsdPerMTok":0.00875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"amazon.nova-pro-v1:0","displayName":"Nova Pro","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.8,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-fable-5","displayName":"Claude Fable 5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-fable-5-1","displayName":"Claude Fable 5.1","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-4-1-20250805-v1:0","displayName":"Claude Opus 4.1","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-4-5-20251101-v1:0","displayName":"Claude Opus 4.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-4-6-v1","displayName":"Claude Opus 4.6","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-4-7","displayName":"Claude Opus 4.7","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-opus-5","displayName":"Claude Opus 5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-sonnet-4-6","displayName":"Claude Sonnet 4.6","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5 (AU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-opus-4-6-v1","displayName":"AU Anthropic Claude Opus 4.6","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":16.5,"outputUsdPerMTok":82.5,"cacheReadUsdPerMTok":1.65,"cacheWriteUsdPerMTok":20.625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8 (AU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-opus-5","displayName":"Claude Opus 5 (AU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5 (AU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-sonnet-4-6","displayName":"AU Anthropic Claude Sonnet 4.6","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3.3,"outputUsdPerMTok":16.5,"cacheReadUsdPerMTok":0.33,"cacheWriteUsdPerMTok":4.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"au.anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5 (AU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"deepseek.r1-v1:0","displayName":"DeepSeek-R1","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.35,"outputUsdPerMTok":5.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"deepseek.v3-v1:0","displayName":"DeepSeek-V3.1","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":81920,"cost":{"inputUsdPerMTok":0.58,"outputUsdPerMTok":1.68},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"deepseek.v3.2","displayName":"DeepSeek-V3.2","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":81920,"cost":{"inputUsdPerMTok":0.62,"outputUsdPerMTok":1.85},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-fable-5","displayName":"Claude Fable 5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":11,"outputUsdPerMTok":55,"cacheReadUsdPerMTok":1.1,"cacheWriteUsdPerMTok":13.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":5.5,"cacheReadUsdPerMTok":0.11,"cacheWriteUsdPerMTok":1.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-opus-4-5-20251101-v1:0","displayName":"Claude Opus 4.5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":27.5,"cacheReadUsdPerMTok":0.55,"cacheWriteUsdPerMTok":6.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-opus-4-6-v1","displayName":"Claude Opus 4.6 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":27.5,"cacheReadUsdPerMTok":0.55,"cacheWriteUsdPerMTok":6.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-opus-4-7","displayName":"Claude Opus 4.7 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":27.5,"cacheReadUsdPerMTok":0.55,"cacheWriteUsdPerMTok":6.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":27.5,"cacheReadUsdPerMTok":0.55,"cacheWriteUsdPerMTok":6.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-opus-5","displayName":"Claude Opus 5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":27.5,"cacheReadUsdPerMTok":0.55,"cacheWriteUsdPerMTok":6.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3.3,"outputUsdPerMTok":16.5,"cacheReadUsdPerMTok":0.33,"cacheWriteUsdPerMTok":4.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-sonnet-4-6","displayName":"Claude Sonnet 4.6 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3.3,"outputUsdPerMTok":16.5,"cacheReadUsdPerMTok":0.33,"cacheWriteUsdPerMTok":4.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"eu.anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5 (EU)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.2,"outputUsdPerMTok":11,"cacheReadUsdPerMTok":0.22,"cacheWriteUsdPerMTok":2.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-fable-5","displayName":"Claude Fable 5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-fable-5-1","displayName":"Claude Fable 5.1 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-opus-4-5-20251101-v1:0","displayName":"Claude Opus 4.5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-opus-4-6-v1","displayName":"Claude Opus 4.6 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-opus-4-7","displayName":"Claude Opus 4.7 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-opus-5","displayName":"Claude Opus 5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-sonnet-4-6","displayName":"Claude Sonnet 4.6 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5 (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"global.openai.gpt-5.6-luna","displayName":"GPT-5.6 Luna (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"global.openai.gpt-5.6-sol","displayName":"GPT-5.6 Sol (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.8,"cacheWriteUsdPerMTok":10}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"global.openai.gpt-5.6-terra","displayName":"GPT-5.6 Terra (Global)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"google.gemma-3-27b-it","displayName":"Google Gemma 3 27B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":202752,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.12,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"google.gemma-3-4b-it","displayName":"Gemma 3 4B IT","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.04,"outputUsdPerMTok":0.08},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-opus-4-7","displayName":"Claude Opus 4.7 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-opus-5","displayName":"Claude Opus 5 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-sonnet-4-6","displayName":"Claude Sonnet 4.6 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"jp.anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5 (JP)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"meta.llama3-1-70b-instruct-v1:0","displayName":"Llama 3.1 70B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.72,"outputUsdPerMTok":0.72},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"meta.llama3-1-8b-instruct-v1:0","displayName":"Llama 3.1 8B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.22},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"meta.llama3-3-70b-instruct-v1:0","displayName":"Llama 3.3 70B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.72,"outputUsdPerMTok":0.72},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"meta.llama4-maverick-17b-instruct-v1:0","displayName":"Llama 4 Maverick 17B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.24,"outputUsdPerMTok":0.97},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"meta.llama4-scout-17b-instruct-v1:0","displayName":"Llama 4 Scout 17B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":3500000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.17,"outputUsdPerMTok":0.66},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"minimax.minimax-m2","displayName":"MiniMax M2","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204608,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"minimax.minimax-m2.1","displayName":"MiniMax M2.1","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"minimax.minimax-m2.5","displayName":"MiniMax M2.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":196608,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"mistral.devstral-2-123b","displayName":"Devstral 2 123B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.magistral-small-2509","displayName":"Magistral Small 1.2","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":128000,"maxOutputTokens":40000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.ministral-3-14b-instruct","displayName":"Ministral 14B 3.0","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.ministral-3-3b-instruct","displayName":"Ministral 3 3B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.ministral-3-8b-instruct","displayName":"Ministral 3 8B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.mistral-large-3-675b-instruct","displayName":"Mistral Large 3","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":256000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.pixtral-large-2502-v1:0","displayName":"Pixtral Large (25.02)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"mistral.voxtral-mini-3b-2507","displayName":"Voxtral Mini 3B 2507","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.04,"outputUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"mistral.voxtral-small-24b-2507","displayName":"Voxtral Small 24B 2507","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":32000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.35},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"moonshot.kimi-k2-thinking","displayName":"Kimi K2 Thinking","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262143,"maxOutputTokens":16000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"moonshotai.kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262143,"maxOutputTokens":16000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"nvidia.nemotron-nano-12b-v2","displayName":"NVIDIA Nemotron Nano 12B v2 VL BF16","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"nvidia.nemotron-nano-3-30b","displayName":"NVIDIA Nemotron Nano 3 30B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"nvidia.nemotron-nano-9b-v2","displayName":"NVIDIA Nemotron Nano 9B v2","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.23},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"nvidia.nemotron-super-3-120b","displayName":"NVIDIA Nemotron 3 Super 120B A12B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.65},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-5.4","displayName":"GPT-5.4","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":272000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.75,"outputUsdPerMTok":16.5,"cacheReadUsdPerMTok":0.275},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-5.5","displayName":"GPT-5.5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":272000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5.5,"outputUsdPerMTok":33,"cacheReadUsdPerMTok":0.55},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":1.32,"cacheReadUsdPerMTok":0.022,"cacheWriteUsdPerMTok":0.275,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.44,"outputUsdPerMTok":1.98,"cacheReadUsdPerMTok":0.044,"cacheWriteUsdPerMTok":0.55}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-5.6-sol","displayName":"GPT-5.6 Sol","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4.4,"outputUsdPerMTok":22,"cacheReadUsdPerMTok":0.44,"cacheWriteUsdPerMTok":5.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8.8,"outputUsdPerMTok":33,"cacheReadUsdPerMTok":0.88,"cacheWriteUsdPerMTok":11}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-5.6-terra","displayName":"GPT-5.6 Terra","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.2,"outputUsdPerMTok":13.2,"cacheReadUsdPerMTok":0.22,"cacheWriteUsdPerMTok":2.75,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4.4,"outputUsdPerMTok":19.8,"cacheReadUsdPerMTok":0.44,"cacheWriteUsdPerMTok":5.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-120b","displayName":"gpt-oss-120b","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-120b-1:0","displayName":"gpt-oss-120b","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-20b","displayName":"gpt-oss-20b","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-20b-1:0","displayName":"gpt-oss-20b","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-safeguard-120b","displayName":"GPT OSS Safeguard 120B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"openai.gpt-oss-safeguard-20b","displayName":"GPT OSS Safeguard 20B","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-235b-a22b-2507-v1:0","displayName":"Qwen3 235B A22B 2507","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.88},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-32b-v1:0","displayName":"Qwen3 32B (dense)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":16384,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-coder-30b-a3b-v1:0","displayName":"Qwen3 Coder 30B A3B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-coder-480b-a35b-v1:0","displayName":"Qwen3 Coder 480B A35B Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":1.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-coder-next","displayName":"Qwen3 Coder Next","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":1.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-next-80b-a3b","displayName":"Qwen/Qwen3-Next-80B-A3B-Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":1.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"qwen.qwen3-vl-235b-a22b","displayName":"Qwen/Qwen3-VL-235B-A22B-Instruct","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-fable-5","displayName":"Claude Fable 5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-fable-5-1","displayName":"Claude Fable 5.1 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":11,"outputUsdPerMTok":55,"cacheReadUsdPerMTok":0.275,"cacheWriteUsdPerMTok":13.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-haiku-4-5-20251001-v1:0","displayName":"Claude Haiku 4.5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-4-1-20250805-v1:0","displayName":"Claude Opus 4.1 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-4-5-20251101-v1:0","displayName":"Claude Opus 4.5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-4-6-v1","displayName":"Claude Opus 4.6 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-4-7","displayName":"Claude Opus 4.7 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-4-8","displayName":"Claude Opus 4.8 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-opus-5","displayName":"Claude Opus 5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-sonnet-4-5-20250929-v1:0","displayName":"Claude Sonnet 4.5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-sonnet-4-6","displayName":"Claude Sonnet 4.6 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.anthropic.claude-sonnet-5","displayName":"Claude Sonnet 5 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true,"supportsPromptCacheMarkers":true,"supportsThinkingSignatures":true,"forceAdaptiveThinking":true}}, + {"providerId":"amazon-bedrock","modelId":"us.deepseek.r1-v1:0","displayName":"DeepSeek-R1 (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.35,"outputUsdPerMTok":5.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"us.meta.llama4-maverick-17b-instruct-v1:0","displayName":"Llama 4 Maverick 17B Instruct (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.24,"outputUsdPerMTok":0.97},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"us.meta.llama4-scout-17b-instruct-v1:0","displayName":"Llama 4 Scout 17B Instruct (US)","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":3500000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.17,"outputUsdPerMTok":0.66},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"writer.palmyra-x4-v1:0","displayName":"Palmyra X4","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":122880,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"writer.palmyra-x5-v1:0","displayName":"Palmyra X5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1040000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream"}}, + {"providerId":"amazon-bedrock","modelId":"xai.grok-4.3","displayName":"Grok 4.3","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"xai.grok-4.6","displayName":"Grok 4.6","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2.2,"outputUsdPerMTok":6.6,"cacheReadUsdPerMTok":0.55},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"zai.glm-4.7","displayName":"GLM-4.7","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"zai.glm-4.7-flash","displayName":"GLM-4.7-Flash","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, + {"providerId":"amazon-bedrock","modelId":"zai.glm-5","displayName":"GLM-5","apiDialect":"bedrock-converse-stream","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":101376,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://bedrock-runtime.{region}.amazonaws.com","variables":[{"name":"region","setting":"region","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"bedrock-converse-stream","supportsStrictTools":true}}, +]; diff --git a/packages/ai/src/catalog.generated/ant-ling.generated.ts b/packages/ai/src/catalog.generated/ant-ling.generated.ts new file mode 100644 index 0000000..f8e922e --- /dev/null +++ b/packages/ai/src/catalog.generated/ant-ling.generated.ts @@ -0,0 +1,13 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"ant-ling","modelId":"Ling-2.6-1T","displayName":"Ling 2.6 1T","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":32000,"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.ant-ling.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"}}, + {"providerId":"ant-ling","modelId":"Ling-2.6-flash","displayName":"Ling 2.6 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":32000,"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.ant-ling.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"}}, + {"providerId":"ant-ling","modelId":"Ling-3.0-flash","displayName":"Ling 3.0 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":32000,"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.ant-ling.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"}}, + {"providerId":"ant-ling","modelId":"Ring-2.6-1T","displayName":"Ring 2.6 1T","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":32000,"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.ant-ling.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"ant-ling"}}, +]; diff --git a/packages/ai/src/catalog.generated/anthropic.generated.ts b/packages/ai/src/catalog.generated/anthropic.generated.ts new file mode 100644 index 0000000..5c5b27e --- /dev/null +++ b/packages/ai/src/catalog.generated/anthropic.generated.ts @@ -0,0 +1,23 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"anthropic","modelId":"claude-fable-5","displayName":"Claude Fable 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true,"forceAdaptiveThinking":true}}, + {"providerId":"anthropic","modelId":"claude-fable-5-1","displayName":"Claude Fable 5.1","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true,"forceAdaptiveThinking":true}}, + {"providerId":"anthropic","modelId":"claude-haiku-4-5","displayName":"Claude Haiku 4.5 (latest)","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-haiku-4-5-20251001","displayName":"Claude Haiku 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-opus-4-5","displayName":"Claude Opus 4.5 (latest)","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-opus-4-5-20251101","displayName":"Claude Opus 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-opus-4-6","displayName":"Claude Opus 4.6","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-opus-4-7","displayName":"Claude Opus 4.7","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-opus-4-8","displayName":"Claude Opus 4.8","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true,"forceAdaptiveThinking":true}}, + {"providerId":"anthropic","modelId":"claude-opus-5","displayName":"Claude Opus 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true,"forceAdaptiveThinking":true}}, + {"providerId":"anthropic","modelId":"claude-sonnet-4-5","displayName":"Claude Sonnet 4.5 (latest)","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-sonnet-4-5-20250929","displayName":"Claude Sonnet 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-sonnet-4-6","displayName":"Claude Sonnet 4.6","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"anthropic","modelId":"claude-sonnet-5","displayName":"Claude Sonnet 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.anthropic.com"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsLongCacheRetention":true,"supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true,"forceAdaptiveThinking":true}}, +]; diff --git a/packages/ai/src/catalog.generated/azure-openai-responses.generated.ts b/packages/ai/src/catalog.generated/azure-openai-responses.generated.ts new file mode 100644 index 0000000..aa69990 --- /dev/null +++ b/packages/ai/src/catalog.generated/azure-openai-responses.generated.ts @@ -0,0 +1,75 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"azure-openai-responses","modelId":"claude-fable-5","displayName":"Claude Fable 5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-fable-5-1","displayName":"Claude Fable 5.1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-haiku-4-5","displayName":"Claude Haiku 4.5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-mythos-5","displayName":"Claude Mythos 5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-4-1","displayName":"Claude Opus 4.1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-4-5","displayName":"Claude Opus 4.5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-4-6","displayName":"Claude Opus 4.6","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":10,"outputUsdPerMTok":37.5,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-4-7","displayName":"Claude Opus 4.7","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-4-8","displayName":"Claude Opus 4.8","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":10,"outputUsdPerMTok":37.5,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-opus-5","displayName":"Claude Opus 5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-sonnet-4-5","displayName":"Claude Sonnet 4.5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-sonnet-4-6","displayName":"Claude Sonnet 4.6","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"claude-sonnet-5","displayName":"Claude Sonnet 5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"codestral-2501","displayName":"Codestral 25.01","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"codex-mini","displayName":"Codex Mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"cohere-command-a","displayName":"Command A","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"deepseek-v3.2","displayName":"DeepSeek-V3.2","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.58,"outputUsdPerMTok":1.68},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4-turbo","displayName":"GPT-4 Turbo","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4-turbo-vision","displayName":"GPT-4 Turbo Vision","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4.1","displayName":"GPT-4.1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4.1-mini","displayName":"GPT-4.1 mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4.1-nano","displayName":"GPT-4.1 nano","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4o","displayName":"GPT-4o","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-4o-mini","displayName":"GPT-4o mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5","displayName":"GPT-5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5-codex","displayName":"GPT-5-Codex","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5-mini","displayName":"GPT-5 Mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5-nano","displayName":"GPT-5 Nano","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5-pro","displayName":"GPT-5 Pro","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":272000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":120},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.1","displayName":"GPT-5.1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.1-codex","displayName":"GPT-5.1 Codex","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.1-codex-max","displayName":"GPT-5.1 Codex Max","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.1-codex-mini","displayName":"GPT-5.1 Codex Mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.2","displayName":"GPT-5.2","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.2-codex","displayName":"GPT-5.2 Codex","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.3-codex","displayName":"GPT-5.3 Codex","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.4","displayName":"GPT-5.4","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.4-mini","displayName":"GPT-5.4 Mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.4-nano","displayName":"GPT-5.4 Nano","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.4-pro","displayName":"GPT-5.4 Pro","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":60,"outputUsdPerMTok":270}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.5","displayName":"GPT-5.5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":10,"outputUsdPerMTok":45,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.6-sol","displayName":"GPT-5.6 Sol","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":10,"outputUsdPerMTok":45,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-5.6-terra","displayName":"GPT-5.6 Terra","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"gpt-chat-latest","displayName":"GPT Chat Latest","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"grok-4-1-fast-non-reasoning","displayName":"Grok 4.1 Fast (Non-Reasoning)","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"grok-4-1-fast-reasoning","displayName":"Grok 4.1 Fast (Reasoning)","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"grok-4-20-non-reasoning","displayName":"Grok 4.20 (Non-Reasoning)","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"grok-4-20-reasoning","displayName":"Grok 4.20 (Reasoning)","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"grok-4.6","displayName":"Grok 4.6","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":200000,"maxOutputTokens":128000,"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"llama-3.3-70b-instruct","displayName":"Llama-3.3-70B-Instruct","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.71,"outputUsdPerMTok":0.71},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"llama-4-maverick-17b-128e-instruct-fp8","displayName":"Llama 4 Maverick 17B 128E Instruct FP8","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"llama-4-scout-17b-16e-instruct","displayName":"Llama 4 Scout 17B 16E Instruct","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.78},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"ministral-3b","displayName":"Ministral 3B","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.04,"outputUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"mistral-medium-2505","displayName":"Mistral Medium 3","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"mistral-small-2503","displayName":"Mistral Small 3.1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"model-router","displayName":"Model Router","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":200000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"o1","displayName":"o1","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":60,"cacheReadUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"o3","displayName":"o3","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"o3-mini","displayName":"o3-mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.55},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"o4-mini","displayName":"o4-mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.275},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"phi-4-mini","displayName":"Phi-4-mini","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"azure-openai-responses","modelId":"phi-4-mini-reasoning","displayName":"Phi-4-mini-reasoning","apiDialect":"azure-openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{resource}.openai.azure.com/openai/v1","variables":[{"name":"resource","setting":"resource","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"azure-openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, +]; diff --git a/packages/ai/src/catalog.generated/baseten.generated.ts b/packages/ai/src/catalog.generated/baseten.generated.ts new file mode 100644 index 0000000..64aa735 --- /dev/null +++ b/packages/ai/src/catalog.generated/baseten.generated.ts @@ -0,0 +1,31 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"baseten","modelId":"deepseek-ai/DeepSeek-V3.1","displayName":"DeepSeek V3.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":164000,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"deepseek-ai/DeepSeek-V4-Flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.13,"outputUsdPerMTok":0.26,"cacheReadUsdPerMTok":0.028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"deepseek-ai/DeepSeek-V4-Pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.74,"outputUsdPerMTok":3.48,"cacheReadUsdPerMTok":0.145},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"deepseek-ai/DeepSeek-V4-Pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.32,"outputUsdPerMTok":3.96},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"MiniMaxAI/MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204000,"maxOutputTokens":204000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"moonshotai/Kimi-K2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"moonshotai/Kimi-K2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"moonshotai/Kimi-K2.7-Code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"moonshotai/Kimi-K3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"nvidia/Nemotron-120B-A12B","displayName":"Nemotron Super","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":202800,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.75,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B","displayName":"Nemotron Ultra","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":202800,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"openai/gpt-oss-120b","displayName":"OpenAI GPT 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":128072,"maxOutputTokens":128072,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"thinkingmachines/inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":4.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"thinkingmachines/inkling-small","displayName":"Inkling Small","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-4.7","displayName":"GLM 4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":200000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5","displayName":"GLM 5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":202800,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":3.15,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.1","displayName":"GLM 5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":202800,"cost":{"inputUsdPerMTok":1.3,"outputUsdPerMTok":4.3,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.2","displayName":"GLM 5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.2-Fast","displayName":"GLM 5.2 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2.1,"outputUsdPerMTok":6.6,"cacheReadUsdPerMTok":0.21},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.3","displayName":"GLM 5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.14},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.3-Fast","displayName":"GLM 5.3 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2.1,"outputUsdPerMTok":6.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, + {"providerId":"baseten","modelId":"zai-org/GLM-5.3-Flash","displayName":"GLM 5.3 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://inference.baseten.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"baseten"}}, +]; diff --git a/packages/ai/src/catalog.generated/cerebras.generated.ts b/packages/ai/src/catalog.generated/cerebras.generated.ts new file mode 100644 index 0000000..2268d9f --- /dev/null +++ b/packages/ai/src/catalog.generated/cerebras.generated.ts @@ -0,0 +1,11 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"cerebras","modelId":"gemma-4-31b","displayName":"Gemma 4 31B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":131072,"maxOutputTokens":40960,"cost":{"inputUsdPerMTok":0.99,"outputUsdPerMTok":1.49},"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.cerebras.ai/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cerebras","modelId":"gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":40960,"cost":{"inputUsdPerMTok":0.35,"outputUsdPerMTok":0.75},"cache":{"supported":false,"defaultRetention":"none","supportedRetentions":["none"]},"endpoint":{"type":"fixed","baseUrl":"https://api.cerebras.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/cloudflare-workers-ai.generated.ts b/packages/ai/src/catalog.generated/cloudflare-workers-ai.generated.ts new file mode 100644 index 0000000..fa25268 --- /dev/null +++ b/packages/ai/src/catalog.generated/cloudflare-workers-ai.generated.ts @@ -0,0 +1,27 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"cloudflare-workers-ai","modelId":"@cf/deepseek-ai/deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1310720,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.44,"outputUsdPerMTok":1.32,"cacheReadUsdPerMTok":0.014},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/deepseek-ai/deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1.32,"outputUsdPerMTok":3.96,"cacheReadUsdPerMTok":0.044},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/google/gemma-4-26b-a4b-it","displayName":"Gemma 4 26B A4B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/ibm-granite/granite-4.0-h-micro","displayName":"Granite 4.0 H Micro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131000,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.017,"outputUsdPerMTok":0.112},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/meta/llama-3.3-70b-instruct-fp8-fast","displayName":"Llama 3.3 70B Instruct fp8 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":24000,"maxOutputTokens":24000,"cost":{"inputUsdPerMTok":0.293,"outputUsdPerMTok":2.253},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/meta/llama-4-scout-17b-16e-instruct","displayName":"Llama 4 Scout 17B 16E Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":131000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.27,"outputUsdPerMTok":0.85},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/mistralai/mistral-small-3.1-24b-instruct","displayName":"Mistral Small 3.1 24B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.351,"outputUsdPerMTok":0.555},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/moonshotai/kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/moonshotai/kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/nvidia/nemotron-3-120b-a12b","displayName":"Nemotron 3 Super 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/openai/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.35,"outputUsdPerMTok":0.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/qwen/qwen3-30b-a3b-fp8","displayName":"Qwen3 30B A3b fp8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.0509,"outputUsdPerMTok":0.335},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/qwen/qwen3.8-27b","displayName":"Qwen3.8 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.45,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/zai-org/glm-4.7-flash","displayName":"GLM-4.7-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.0605,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/zai-org/glm-5.2","displayName":"Glm 5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/zai-org/glm-5.3","displayName":"Glm 5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1310720,"maxOutputTokens":1310720,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"cloudflare-workers-ai","modelId":"@cf/zai-org/glm-5.3-flash","displayName":"Glm 5.3 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1310720,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1","variables":[{"name":"account","setting":"account","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/deepseek.generated.ts b/packages/ai/src/catalog.generated/deepseek.generated.ts new file mode 100644 index 0000000..d15498f --- /dev/null +++ b/packages/ai/src/catalog.generated/deepseek.generated.ts @@ -0,0 +1,12 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"deepseek","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.deepseek.com"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}}, + {"providerId":"deepseek","modelId":"deepseek-v4-flash-vision-exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.deepseek.com"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}}, + {"providerId":"deepseek","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.003625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.deepseek.com"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"requiresReasoningContentOnAssistantMessages":true,"thinkingFormat":"deepseek"}}, +]; diff --git a/packages/ai/src/catalog.generated/fireworks.generated.ts b/packages/ai/src/catalog.generated/fireworks.generated.ts new file mode 100644 index 0000000..83016b7 --- /dev/null +++ b/packages/ai/src/catalog.generated/fireworks.generated.ts @@ -0,0 +1,29 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"fireworks","modelId":"accounts/fireworks/models/deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/deepseek-v4-flash-vision-exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":1.32,"outputUsdPerMTok":3.96,"cacheReadUsdPerMTok":0.044},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/glm-5p2","displayName":"GLM 5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048575,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.14},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/glm-5p3","displayName":"GLM 5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/glm-5p3-flash","displayName":"GLM 5.3 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.015},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":4.05,"cacheReadUsdPerMTok":0.17},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/kimi-k2p6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/kimi-k2p7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/minimax-m3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":512000,"maxOutputTokens":512000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/muse-glimmer-30b","displayName":"Muse Glimmer 30B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.35,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/nemotron-3-ultra-nvfp4","displayName":"Nemotron 3 Ultra 550B A55B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.119},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/nemotron-lightning-3p5-30b-a3b","displayName":"Nemotron 3.5 Lightning 30B A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/qwen3p7-plus","displayName":"Qwen 3.7 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.08},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/qwen3p8-2p4t-a95b","displayName":"Qwen3.8 2.4T A95B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/models/qwen3p8-max","displayName":"Qwen3.8 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/routers/glm-5p2-fast","displayName":"GLM 5.2 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048575,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":2.1,"outputUsdPerMTok":6.6,"cacheReadUsdPerMTok":0.21},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"fireworks","modelId":"accounts/fireworks/routers/kimi-k3-fast","displayName":"Kimi K3 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":4.5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.45},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.fireworks.ai/inference/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/google-vertex.generated.ts b/packages/ai/src/catalog.generated/google-vertex.generated.ts new file mode 100644 index 0000000..4237060 --- /dev/null +++ b/packages/ai/src/catalog.generated/google-vertex.generated.ts @@ -0,0 +1,48 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"google-vertex","modelId":"claude-fable-5-1@default","displayName":"Claude Fable 5.1","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-fable-5@default","displayName":"Claude Fable 5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-haiku-4-5@20251001","displayName":"Claude Haiku 4.5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4-1@20250805","displayName":"Claude Opus 4.1","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4-5@20251101","displayName":"Claude Opus 4.5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4-6@default","displayName":"Claude Opus 4.6","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":10,"outputUsdPerMTok":37.5,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4-7@default","displayName":"Claude Opus 4.7","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":10,"outputUsdPerMTok":37.5,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4-8@default","displayName":"Claude Opus 4.8","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":10,"outputUsdPerMTok":37.5,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-4@20250514","displayName":"Claude Opus 4","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-opus-5@default","displayName":"Claude Opus 5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-sonnet-4-5@20250929","displayName":"Claude Sonnet 4.5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-sonnet-4-6@default","displayName":"Claude Sonnet 4.6","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":6,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.6,"cacheWriteUsdPerMTok":7.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-sonnet-4@20250514","displayName":"Claude Sonnet 4","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"claude-sonnet-5@default","displayName":"Claude Sonnet 5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"deepseek-ai/deepseek-v3.1-maas","displayName":"DeepSeek V3.1","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":1.7,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"deepseek-ai/deepseek-v3.2-maas","displayName":"DeepSeek V3.2","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.56,"outputUsdPerMTok":1.68,"cacheReadUsdPerMTok":0.056},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"gemini-2.5-flash","displayName":"Gemini 2.5 Flash","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"gemini-2.5-flash-lite","displayName":"Gemini 2.5 Flash-Lite","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"gemini-2.5-pro","displayName":"Gemini 2.5 Pro","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"gemini-3-flash-preview","displayName":"Gemini 3 Flash Preview","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.1-flash-lite","displayName":"Gemini 3.1 Flash Lite","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.1-flash-lite-preview","displayName":"Gemini 3.1 Flash Lite Preview","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.1-pro-preview","displayName":"Gemini 3.1 Pro Preview","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.1-pro-preview-customtools","displayName":"Gemini 3.1 Pro Preview Custom Tools","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.5-flash","displayName":"Gemini 3.5 Flash","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.5-flash-lite","displayName":"Gemini 3.5 Flash Lite","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.6-flash","displayName":"Gemini 3.6 Flash","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.7-flash","displayName":"Gemini 3.7 Flash","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-3.8-flash","displayName":"Gemini 3.8 Flash","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex","supportsStrictTools":true}}, + {"providerId":"google-vertex","modelId":"gemini-flash-latest","displayName":"Gemini Flash Latest","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"gemini-flash-lite-latest","displayName":"Gemini Flash-Lite Latest","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"meta/llama-3.3-70b-instruct-maas","displayName":"Llama 3.3 70B Instruct","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.72,"outputUsdPerMTok":0.72},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"meta/llama-4-maverick-17b-128e-instruct-maas","displayName":"Llama 4 Maverick 17B 128E Instruct","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":524288,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.35,"outputUsdPerMTok":1.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"moonshotai/kimi-k2-thinking-maas","displayName":"Kimi K2 Thinking","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"openai/gpt-oss-120b-maas","displayName":"GPT OSS 120B","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.09,"outputUsdPerMTok":0.36},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"available"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"openai/gpt-oss-20b-maas","displayName":"GPT OSS 20B","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.25,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"qwen/qwen3-235b-a22b-instruct-2507-maas","displayName":"Qwen3 235B A22B Instruct","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.88},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"zai-org/glm-4.7-maas","displayName":"GLM-4.7","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, + {"providerId":"google-vertex","modelId":"zai-org/glm-5-maas","displayName":"GLM-5","apiDialect":"google-vertex","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"template","template":"https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}","variables":[{"name":"location","setting":"location","required":true},{"name":"project","setting":"project","required":true}]},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-vertex"}}, +]; diff --git a/packages/ai/src/catalog.generated/google.generated.ts b/packages/ai/src/catalog.generated/google.generated.ts new file mode 100644 index 0000000..1c53a56 --- /dev/null +++ b/packages/ai/src/catalog.generated/google.generated.ts @@ -0,0 +1,31 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"google","modelId":"deep-research-max-preview-04-2026","displayName":"Deep Research Max Preview (Apr-21-2026)","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"deep-research-preview-04-2026","displayName":"Deep Research Preview (Apr-21-2026)","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-2.5-computer-use-preview-10-2025","displayName":"Gemini 2.5 Computer Use Preview 10-2025","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":15}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-2.5-flash","displayName":"Gemini 2.5 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-2.5-flash-lite","displayName":"Gemini 2.5 Flash-Lite","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-2.5-pro","displayName":"Gemini 2.5 Pro","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-3-flash-preview","displayName":"Gemini 3 Flash Preview","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-flash-lite","displayName":"Gemini 3.1 Flash Lite","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-flash-lite-image","displayName":"Nano Banana 2 Lite","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":65536,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-flash-lite-preview","displayName":"Gemini 3.1 Flash Lite Preview","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-flash-live-preview","displayName":"Gemini 3.1 Flash Live Preview","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-pro-preview","displayName":"Gemini 3.1 Pro Preview","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.1-pro-preview-customtools","displayName":"Gemini 3.1 Pro Preview Custom Tools","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.5-flash","displayName":"Gemini 3.5 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.5-flash-lite","displayName":"Gemini 3.5 Flash Lite","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.6-flash","displayName":"Gemini 3.6 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.7-flash","displayName":"Gemini 3.7 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-3.8-flash","displayName":"Gemini 3.8 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai","supportsStrictTools":true}}, + {"providerId":"google","modelId":"gemini-flash-latest","displayName":"Gemini Flash Latest","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemini-flash-lite-latest","displayName":"Gemini Flash-Lite Latest","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemma-4-26b-a4b-it","displayName":"Gemma 4 26B A4B IT","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":32768,"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"google","modelId":"gemma-4-31b-it","displayName":"Gemma 4 31B IT","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":32768,"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://generativelanguage.googleapis.com/v1beta"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, +]; diff --git a/packages/ai/src/catalog.generated/groq.generated.ts b/packages/ai/src/catalog.generated/groq.generated.ts new file mode 100644 index 0000000..7f88df1 --- /dev/null +++ b/packages/ai/src/catalog.generated/groq.generated.ts @@ -0,0 +1,16 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"groq","modelId":"llama-3.1-8b-instant","displayName":"Llama 3.1 8B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.08},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"llama-3.3-70b-versatile","displayName":"Llama 3.3 70B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.59,"outputUsdPerMTok":0.79},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"openai/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.3,"cacheReadUsdPerMTok":0.0375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"openai/gpt-oss-safeguard-20b","displayName":"Safety GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"qwen/qwen3.6-27b","displayName":"Qwen3.6 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":null,"xhigh":null,"max":null,"off":"none"},"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"groq","modelId":"qwen/qwen3.8-27b","displayName":"Qwen3.8 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":131042,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.8,"outputUsdPerMTok":4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.groq.com/openai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/huggingface.generated.ts b/packages/ai/src/catalog.generated/huggingface.generated.ts new file mode 100644 index 0000000..e266eea --- /dev/null +++ b/packages/ai/src/catalog.generated/huggingface.generated.ts @@ -0,0 +1,79 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-R1","displayName":"DeepSeek-R1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":64000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.7,"outputUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-R1-0528","displayName":"DeepSeek-R1-0528","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":163840,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V3","displayName":"DeepSeek-V3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":64000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V3-0324","displayName":"DeepSeek V3 0324","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":163840,"maxOutputTokens":163840,"cost":{"inputUsdPerMTok":0.27,"outputUsdPerMTok":1.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V3.1","displayName":"DeepSeek-V3.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.27,"outputUsdPerMTok":1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V3.2","displayName":"DeepSeek-V3.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":163840,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.28,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V4-Flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V4-Flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V4-Flash-Vision-Exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.44,"outputUsdPerMTok":1.32},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V4-Pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.003625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"deepseek-ai/DeepSeek-V4-Pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":1.32,"outputUsdPerMTok":3.96},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"google/gemma-4-26B-A4B-it","displayName":"Gemma 4 26B A4B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.13,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"google/gemma-4-31B-it","displayName":"Gemma 4 31B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"meta-llama/Llama-3.1-8B-Instruct","displayName":"Llama-3.1-8B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"meta-llama/Llama-3.3-70B-Instruct","displayName":"Llama-3.3-70B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.59,"outputUsdPerMTok":0.79},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"MiniMaxAI/MiniMax-M2","displayName":"MiniMax-M2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"MiniMaxAI/MiniMax-M2.1","displayName":"MiniMax-M2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"MiniMaxAI/MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"MiniMaxAI/MiniMax-M2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"MiniMaxAI/MiniMax-M3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":524288,"maxOutputTokens":512000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2-Instruct","displayName":"Kimi-K2-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2-Instruct-0905","displayName":"Kimi-K2-Instruct-0905","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2-Thinking","displayName":"Kimi-K2-Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2.5","displayName":"Kimi-K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2.6","displayName":"Kimi-K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K2.7-Code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"moonshotai/Kimi-K3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"openai/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":0.69},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen2.5-Coder-32B-Instruct","displayName":"Qwen2.5-Coder-32B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-235B-A22B","displayName":"Qwen3 235B-A22B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":40960,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-235B-A22B-Instruct-2507","displayName":"Qwen3 235B-A22B Instruct 2507","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.855,"outputUsdPerMTok":2.565},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-235B-A22B-Thinking-2507","displayName":"Qwen3-235B-A22B-Thinking-2507","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-30B-A3B","displayName":"Qwen3 30B A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":40960,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.12,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-32B","displayName":"Qwen3 32B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.29,"outputUsdPerMTok":0.59},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-Coder-30B-A3B-Instruct","displayName":"Qwen3-Coder 30B-A3B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-Coder-480B-A35B-Instruct","displayName":"Qwen3-Coder-480B-A35B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-Coder-Next","displayName":"Qwen3-Coder-Next","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-Next-80B-A3B-Instruct","displayName":"Qwen3-Next-80B-A3B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-Next-80B-A3B-Thinking","displayName":"Qwen3-Next-80B-A3B-Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-VL-235B-A22B-Instruct","displayName":"Qwen3 VL 235B A22B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3-VL-235B-A22B-Thinking","displayName":"Qwen3 VL 235B A22B Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.98,"outputUsdPerMTok":3.95},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.5-122B-A10B","displayName":"Qwen3.5 122B-A10B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":3.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.5-27B","displayName":"Qwen3.5 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.5-35B-A3B","displayName":"Qwen3.5 35B-A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.5-397B-A17B","displayName":"Qwen3.5-397B-A17B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.5-9B","displayName":"Qwen3.5 9B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.17,"outputUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.6-27B","displayName":"Qwen3.6 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.47,"outputUsdPerMTok":3.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.6-35B-A3B","displayName":"Qwen3.6 35B-A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.95},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.8-2.4T-A95B","displayName":"Qwen3.8 2.4T A95B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"Qwen/Qwen3.8-27B","displayName":"Qwen3.8 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"stepfun-ai/Step-3.5-Flash","displayName":"Step 3.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"stepfun-ai/Step-3.7-Flash","displayName":"Step 3.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"tencent/Hy3","displayName":"Hy3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.58},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"thinkingmachines/Inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":4.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"XiaomiMiMo/MiMo-V2-Flash","displayName":"MiMo-V2-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"XiaomiMiMo/MiMo-V2.5","displayName":"MiMo-V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"XiaomiMiMo/MiMo-V2.5-Pro","displayName":"MiMo-V2.5-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.5","displayName":"GLM-4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.5-Air","displayName":"GLM-4.5-Air","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":131072,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0.13,"outputUsdPerMTok":0.85},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.5V","displayName":"GLM-4.5V","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":65536,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":1.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.6","displayName":"GLM-4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.55,"outputUsdPerMTok":2.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.6V-Flash","displayName":"GLM-4.6V-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.7","displayName":"GLM-4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-4.7-Flash","displayName":"GLM-4.7-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":200000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"huggingface","modelId":"zai-org/GLM-5.3-Flash","displayName":"GLM-5.3-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://router.huggingface.co/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/kimi-coding.generated.ts b/packages/ai/src/catalog.generated/kimi-coding.generated.ts new file mode 100644 index 0000000..8b9eca9 --- /dev/null +++ b/packages/ai/src/catalog.generated/kimi-coding.generated.ts @@ -0,0 +1,13 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"kimi-coding","modelId":"k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.kimi.com/coding/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"kimi-coding","modelId":"k3-256k","displayName":"Kimi K3-256K","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.kimi.com/coding/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"kimi-coding","modelId":"kimi-for-coding","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.kimi.com/coding/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"kimi-coding","modelId":"kimi-for-coding-highspeed","displayName":"Kimi For Coding HighSpeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.kimi.com/coding/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/minimax-cn.generated.ts b/packages/ai/src/catalog.generated/minimax-cn.generated.ts new file mode 100644 index 0000000..9adeffe --- /dev/null +++ b/packages/ai/src/catalog.generated/minimax-cn.generated.ts @@ -0,0 +1,16 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"minimax-cn","modelId":"MiniMax-M2","displayName":"MiniMax-M2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M2.1","displayName":"MiniMax-M2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M2.5-highspeed","displayName":"MiniMax-M2.5-highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M2.7-highspeed","displayName":"MiniMax-M2.7-highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax-cn","modelId":"MiniMax-M3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":512000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"tiers":[{"inputTokensAbove":512000,"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.12}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimaxi.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/minimax.generated.ts b/packages/ai/src/catalog.generated/minimax.generated.ts new file mode 100644 index 0000000..3958536 --- /dev/null +++ b/packages/ai/src/catalog.generated/minimax.generated.ts @@ -0,0 +1,16 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"minimax","modelId":"MiniMax-M2","displayName":"MiniMax-M2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M2.1","displayName":"MiniMax-M2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M2.5-highspeed","displayName":"MiniMax-M2.5-highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M2.7-highspeed","displayName":"MiniMax-M2.7-highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"minimax","modelId":"MiniMax-M3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":512000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"tiers":[{"inputTokensAbove":512000,"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.12}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.minimax.io/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/mistral.generated.ts b/packages/ai/src/catalog.generated/mistral.generated.ts new file mode 100644 index 0000000..041118f --- /dev/null +++ b/packages/ai/src/catalog.generated/mistral.generated.ts @@ -0,0 +1,40 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"mistral","modelId":"codestral-latest","displayName":"Codestral (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-2512","displayName":"Devstral 2","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-latest","displayName":"Devstral 2","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-medium-2507","displayName":"Devstral Medium","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-medium-latest","displayName":"Devstral 2 (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-small-2505","displayName":"Devstral Small 2505","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"devstral-small-2507","displayName":"Devstral Small","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"labs-devstral-small-2512","displayName":"Devstral Small 2","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"magistral-medium-latest","displayName":"Magistral Medium (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"magistral-small","displayName":"Magistral Small","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"ministral-3b-latest","displayName":"Ministral 3B (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.04,"outputUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"ministral-8b-latest","displayName":"Ministral 8B (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-large-2411","displayName":"Mistral Large 2.1","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-large-2512","displayName":"Mistral Large 3","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-large-latest","displayName":"Mistral Large (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-medium-2505","displayName":"Mistral Medium 3","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-medium-2508","displayName":"Mistral Medium 3.1","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-medium-2604","displayName":"Mistral Medium 3.5","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations","supportsStrictTools":true}}, + {"providerId":"mistral","modelId":"mistral-medium-latest","displayName":"Mistral Medium (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations","supportsStrictTools":true}}, + {"providerId":"mistral","modelId":"mistral-nemo","displayName":"Mistral Nemo","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-small-2506","displayName":"Mistral Small 3.2","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-small-2603","displayName":"Mistral Small 4","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"mistral-small-latest","displayName":"Mistral Small (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"open-mistral-7b","displayName":"Mistral 7B","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":8000,"maxOutputTokens":8000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"open-mistral-nemo","displayName":"Open Mistral Nemo","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"open-mixtral-8x22b","displayName":"Mixtral 8x22B","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":64000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"open-mixtral-8x7b","displayName":"Mixtral 8x7B","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0.7,"outputUsdPerMTok":0.7},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"pixtral-12b","displayName":"Pixtral 12B","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"pixtral-large-latest","displayName":"Pixtral Large (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"voxtral-small-latest","displayName":"Voxtral Small (latest)","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":32000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"mistral-conversations"}}, + {"providerId":"mistral","modelId":"zai-glm-5-2","displayName":"GLM-5.2","apiDialect":"mistral-conversations","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.14},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.mistral.ai/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"mistral-conversations","supportsStrictTools":true}}, +]; diff --git a/packages/ai/src/catalog.generated/moonshotai-cn.generated.ts b/packages/ai/src/catalog.generated/moonshotai-cn.generated.ts new file mode 100644 index 0000000..4cbfd49 --- /dev/null +++ b/packages/ai/src/catalog.generated/moonshotai-cn.generated.ts @@ -0,0 +1,19 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"moonshotai-cn","modelId":"kimi-k2-0711-preview","displayName":"Kimi K2 0711","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2-0905-preview","displayName":"Kimi K2 0905","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2-thinking","displayName":"Kimi K2 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2-thinking-turbo","displayName":"Kimi K2 Thinking Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.15,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2-turbo-preview","displayName":"Kimi K2 Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2.4,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k2.7-code-highspeed","displayName":"Kimi K2.7 Code HighSpeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.9,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.38},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai-cn","modelId":"kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.cn/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/moonshotai.generated.ts b/packages/ai/src/catalog.generated/moonshotai.generated.ts new file mode 100644 index 0000000..10e268f --- /dev/null +++ b/packages/ai/src/catalog.generated/moonshotai.generated.ts @@ -0,0 +1,19 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"moonshotai","modelId":"kimi-k2-0711-preview","displayName":"Kimi K2 0711","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2-0905-preview","displayName":"Kimi K2 0905","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2-thinking","displayName":"Kimi K2 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2-thinking-turbo","displayName":"Kimi K2 Thinking Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.15,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2-turbo-preview","displayName":"Kimi K2 Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2.4,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k2.7-code-highspeed","displayName":"Kimi K2.7 Code HighSpeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.9,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.38},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"moonshotai","modelId":"kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.moonshot.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/nvidia.generated.ts b/packages/ai/src/catalog.generated/nvidia.generated.ts new file mode 100644 index 0000000..ed06980 --- /dev/null +++ b/packages/ai/src/catalog.generated/nvidia.generated.ts @@ -0,0 +1,73 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"nvidia","modelId":"abacusai/dracarys-llama-3.1-70b-instruct","displayName":"dracarys-llama-3.1-70b-instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"bytedance/seed-oss-36b-instruct","displayName":"ByteDance-Seed/Seed-OSS-36B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"deepseek-ai/deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"deepseek-ai/deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"deepseek-ai/deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1048576,"maxOutputTokens":393216,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.003625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"deepseek-ai/deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-2-2b-it","displayName":"Gemma 2 2b It","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-3-12b-it","displayName":"Gemma 3 12B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-3-4b-it","displayName":"Gemma 3 4B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-3n-e2b-it","displayName":"Gemma 3n E2b It","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-3n-e4b-it","displayName":"Gemma 3n E4b It","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"google/gemma-4-31b-it","displayName":"Gemma-4-31B-IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":256000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.1-70b-instruct","displayName":"Llama 3.1 70b Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.1-8b-instruct","displayName":"Llama 3.1 8B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":16000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.2-11b-vision-instruct","displayName":"Llama 3.2 11b Vision Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.2-1b-instruct","displayName":"Llama 3.2 1b Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.2-90b-vision-instruct","displayName":"Llama-3.2-90B-Vision-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-3.3-70b-instruct","displayName":"Llama 3.3 70b Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/llama-4-maverick-17b-128e-instruct","displayName":"Llama 4 Maverick 17b 128e Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"meta/muse-glimmer-30b","displayName":"Muse Glimmer 30B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":"max","off":"none"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"microsoft/phi-4-mini-instruct","displayName":"Phi-4-Mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"minimaxai/minimax-m2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"minimaxai/minimax-m3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/ministral-14b-instruct-2512","displayName":"Ministral 3 14B Instruct 2512","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mistral-7b-instruct-v0.3","displayName":"Mistral-7B-Instruct-v0.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":65536,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mistral-large-3-675b-instruct-2512","displayName":"Mistral Large 3 675B Instruct 2512","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mistral-medium-3.5-128b","displayName":"Mistral Medium 3.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mistral-nemotron","displayName":"mistral-nemotron","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mistral-small-4-119b-2603","displayName":"mistral-small-4-119b-2603","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mixtral-8x22b-instruct","displayName":"Mistral: Mixtral 8x22B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":65536,"maxOutputTokens":13108,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"mistralai/mixtral-8x7b-instruct","displayName":"Mistral: Mixtral 8x7B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":32768,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"moonshotai/kimi-k2-instruct-0905","displayName":"Kimi K2 0905","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"moonshotai/kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"moonshotai/kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/cosmos-reason2-8b","displayName":"Cosmos Reason2 8B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.1-nemotron-70b-instruct","displayName":"Llama 3.1 Nemotron 70B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.1-nemotron-nano-8b-v1","displayName":"Llama 3.1 Nemotron Nano 8B v1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.1-nemotron-nano-vl-8b-v1","displayName":"Llama 3.1 Nemotron Nano VL 8B v1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":32768,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.1-nemotron-ultra-253b-v1","displayName":"Llama 3.1 Nemotron Ultra 253B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.3-nemotron-super-49b-v1","displayName":"Llama 3.3 Nemotron Super 49B v1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/llama-3.3-nemotron-super-49b-v1.5","displayName":"Llama 3.3 Nemotron Super 49B v1.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-3-nano-30b-a3b","displayName":"nemotron-3-nano-30b-a3b","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning","displayName":"Nemotron 3 Nano Omni","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":256000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-3-super-120b-a12b","displayName":"Nemotron 3 Super","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-3-ultra-550b-a55b","displayName":"Nemotron 3 Ultra 550B A55B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-3.5-lightning-30b-a3b","displayName":"Nemotron 3.5 Lightning 30B A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-mini-4b-instruct","displayName":"nemotron-mini-4b-instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-nano-12b-v2-vl","displayName":"Nemotron Nano 12B v2 VL","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nemotron-voicechat","displayName":"nemotron-voicechat","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"nvidia/nvidia-nemotron-nano-9b-v2","displayName":"nvidia-nemotron-nano-9b-v2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"openai/gpt-oss-120b","displayName":"GPT-OSS-120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"poolside/laguna-xs-2.1","displayName":"Laguna XS 2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"qwen/qwen2.5-coder-32b-instruct","displayName":"Qwen2.5 Coder 32b Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"qwen/qwen3-coder-480b-a35b-instruct","displayName":"Qwen3 Coder 480B A35B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":66536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"qwen/qwen3-next-80b-a3b-instruct","displayName":"Qwen3-Next-80B-A3B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"qwen/qwen3.5-122b-a10b","displayName":"Qwen3.5 122B-A10B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"qwen/qwen3.5-397b-a17b","displayName":"Qwen3.5-397B-A17B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"sarvamai/sarvam-m","displayName":"sarvam-m","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"stepfun-ai/step-3.5-flash","displayName":"Step 3.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"stepfun-ai/step-3.7-flash","displayName":"Step 3.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":256000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"thinkingmachines/inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"upstage/solar-10.7b-instruct","displayName":"solar-10.7b-instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"nvidia","modelId":"z-ai/glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://integrate.api.nvidia.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/openai-codex.generated.ts b/packages/ai/src/catalog.generated/openai-codex.generated.ts new file mode 100644 index 0000000..0805c77 --- /dev/null +++ b/packages/ai/src/catalog.generated/openai-codex.generated.ts @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"openai-codex","modelId":"gpt-5","displayName":"GPT-5","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5-mini","displayName":"GPT-5 Mini","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5-nano","displayName":"GPT-5 Nano","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.005},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5-pro","displayName":"GPT-5 Pro","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":272000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":120},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.1","displayName":"GPT-5.1","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.2","displayName":"GPT-5.2","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.2-pro","displayName":"GPT-5.2 Pro","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":21,"outputUsdPerMTok":168},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.3-codex","displayName":"GPT-5.3 Codex","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.3-codex-spark","displayName":"GPT-5.3 Codex Spark","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":128000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.4","displayName":"GPT-5.4","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.4-mini","displayName":"GPT-5.4 mini","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.4-nano","displayName":"GPT-5.4 nano","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.4-pro","displayName":"GPT-5.4 Pro","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":60,"outputUsdPerMTok":270}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.5","displayName":"GPT-5.5","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":10,"outputUsdPerMTok":45,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.5-pro","displayName":"GPT-5.5 Pro","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":60,"outputUsdPerMTok":270}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.6","displayName":"GPT-5.6","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.8,"cacheWriteUsdPerMTok":10}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.6-sol","displayName":"GPT-5.6 Sol","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.8,"cacheWriteUsdPerMTok":10}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai-codex","modelId":"gpt-5.6-terra","displayName":"GPT-5.6 Terra","apiDialect":"openai-codex-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://chatgpt.com/backend-api/codex"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-codex-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, +]; diff --git a/packages/ai/src/catalog.generated/openai.generated.ts b/packages/ai/src/catalog.generated/openai.generated.ts new file mode 100644 index 0000000..b1bb861 --- /dev/null +++ b/packages/ai/src/catalog.generated/openai.generated.ts @@ -0,0 +1,48 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"openai","modelId":"gpt-4","displayName":"GPT-4","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":8192,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":60},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4-turbo","displayName":"GPT-4 Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4.1","displayName":"GPT-4.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4.1-mini","displayName":"GPT-4.1 mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4.1-nano","displayName":"GPT-4.1 nano","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4o","displayName":"GPT-4o","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4o-2024-05-13","displayName":"GPT-4o (2024-05-13)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4o-2024-08-06","displayName":"GPT-4o (2024-08-06)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4o-2024-11-20","displayName":"GPT-4o (2024-11-20)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-4o-mini","displayName":"GPT-4o mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":true,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"openai","modelId":"gpt-5","displayName":"GPT-5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5-mini","displayName":"GPT-5 Mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5-nano","displayName":"GPT-5 Nano","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.005},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5-pro","displayName":"GPT-5 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":272000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":120},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.1","displayName":"GPT-5.1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.2","displayName":"GPT-5.2","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.2-chat-latest","displayName":"GPT-5.2 Chat","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":null,"xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.2-pro","displayName":"GPT-5.2 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":21,"outputUsdPerMTok":168},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.3-chat-latest","displayName":"GPT-5.3 Chat (latest)","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.3-codex","displayName":"GPT-5.3 Codex","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.3-codex-spark","displayName":"GPT-5.3 Codex Spark","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":128000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.4","displayName":"GPT-5.4","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.4-mini","displayName":"GPT-5.4 mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.4-nano","displayName":"GPT-5.4 nano","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.4-pro","displayName":"GPT-5.4 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":60,"outputUsdPerMTok":270}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.5","displayName":"GPT-5.5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":10,"outputUsdPerMTok":45,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.5-pro","displayName":"GPT-5.5 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":60,"outputUsdPerMTok":270}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.6","displayName":"GPT-5.6","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.8,"cacheWriteUsdPerMTok":10}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.6-sol","displayName":"GPT-5.6 Sol","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":8,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.8,"cacheWriteUsdPerMTok":10}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-5.6-terra","displayName":"GPT-5.6 Terra","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-6-astra","displayName":"GPT-6 Astra","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":20,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":2,"cacheWriteUsdPerMTok":25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"gpt-realtime-2.1","displayName":"GPT-Realtime-2.1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":128000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":24,"cacheReadUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o1","displayName":"o1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":60,"cacheReadUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o1-pro","displayName":"o1-pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":150,"outputUsdPerMTok":600},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o3","displayName":"o3","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o3-mini","displayName":"o3-mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.55},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o3-pro","displayName":"o3-pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":20,"outputUsdPerMTok":80},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, + {"providerId":"openai","modelId":"o4-mini","displayName":"o4-mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.275},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short","long"]},"endpoint":{"type":"fixed","baseUrl":"https://api.openai.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsLongCacheRetention":true,"supportsMaxOutputTokens":true}}, +]; diff --git a/packages/ai/src/catalog.generated/opencode-go.generated.ts b/packages/ai/src/catalog.generated/opencode-go.generated.ts new file mode 100644 index 0000000..9b388a5 --- /dev/null +++ b/packages/ai/src/catalog.generated/opencode-go.generated.ts @@ -0,0 +1,44 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"opencode-go","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"deepseek-v4-flash-vision-exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro (New)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.66,"outputUsdPerMTok":1.98,"cacheReadUsdPerMTok":0.022},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"glm-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"glm-5.3-flash","displayName":"GLM-5.3-Flash (2x usage)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.25,"cacheReadUsdPerMTok":0.015},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode-go","modelId":"grok-4.5","displayName":"Grok 4.5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.3,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.6}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode-go","modelId":"grok-4.6","displayName":"Grok 4.6","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode-go","modelId":"hy3","displayName":"Hy3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":256000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.58,"cacheReadUsdPerMTok":0.035},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"hy4-preview","displayName":"Hy4 preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":1024000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.834,"outputUsdPerMTok":2.501,"cacheReadUsdPerMTok":0.042},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":null,"xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"longcat-2.0","displayName":"LongCat-2.0","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.006},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"mimo-v2-omni","displayName":"MiMo V2 Omni","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.08},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"mimo-v2-pro","displayName":"MiMo V2 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":256000,"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"mimo-v2.5","displayName":"MiMo V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"mimo-v2.5-pro","displayName":"MiMo V2.5 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.003625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"minimax-m2.5","displayName":"MiniMax-M2.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"minimax-m2.7","displayName":"MiniMax-M2.7","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"minimax-m3","displayName":"MiniMax-M3","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"tiers":[{"inputTokensAbove":512000,"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.12}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"muse-spark-1.2-contributor","displayName":"Muse Spark 1.2 Contributor","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.002},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode-go","modelId":"muse-spark-1.3-contributor","displayName":"Muse Spark 1.3 Contributor","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.002},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode-go","modelId":"omen-alpha","displayName":"Omen Alpha","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"ox-alpha-free","displayName":"Ox Alpha Free (Unlimited)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode-go","modelId":"qwen3.5-plus","displayName":"Qwen3.5 Plus","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"qwen3.6-plus","displayName":"Qwen3.6 Plus","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05,"cacheWriteUsdPerMTok":0.625,"tiers":[{"inputTokensAbove":256000,"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"qwen3.7-max","displayName":"Qwen3.7 Max","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":3.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"qwen3.7-plus","displayName":"Qwen3.7 Plus","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":256000,"inputUsdPerMTok":1.2,"outputUsdPerMTok":4.8,"cacheReadUsdPerMTok":0.12,"cacheWriteUsdPerMTok":1.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"qwen3.8-flash","displayName":"Qwen3.8 Flash","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.47,"cacheReadUsdPerMTok":0.016,"cacheWriteUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode-go","modelId":"qwen3.8-max","displayName":"Qwen3.8 Max","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/go/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, +]; diff --git a/packages/ai/src/catalog.generated/opencode.generated.ts b/packages/ai/src/catalog.generated/opencode.generated.ts new file mode 100644 index 0000000..ecb8911 --- /dev/null +++ b/packages/ai/src/catalog.generated/opencode.generated.ts @@ -0,0 +1,111 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"opencode","modelId":"big-pickle","displayName":"Big Pickle","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"claude-3-5-haiku","displayName":"Claude Haiku 3.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":200000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.8,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.08,"cacheWriteUsdPerMTok":1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-fable-5","displayName":"Claude Fable 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-fable-5-1","displayName":"Claude Fable 5.1","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-haiku-4-5","displayName":"Claude Haiku 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-4-1","displayName":"Claude Opus 4.1","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-4-5","displayName":"Claude Opus 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-4-6","displayName":"Claude Opus 4.6","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-4-7","displayName":"Claude Opus 4.7","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-4-8","displayName":"Claude Opus 4.8","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-opus-5","displayName":"Claude Opus 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-sonnet-4","displayName":"Claude Sonnet 4","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":6,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.6,"cacheWriteUsdPerMTok":7.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-sonnet-4-5","displayName":"Claude Sonnet 4.5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":6,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.6,"cacheWriteUsdPerMTok":7.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-sonnet-4-6","displayName":"Claude Sonnet 4.6","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"claude-sonnet-5","displayName":"Claude Sonnet 5","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"deepseek-v4-flash-free","displayName":"DeepSeek V4 Flash Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":200000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"deepseek-v4-flash-vision-exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":1.74,"outputUsdPerMTok":3.84,"cacheReadUsdPerMTok":0.145},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"gemini-3-flash","displayName":"Gemini 3 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3-pro","displayName":"Gemini 3 Pro","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.1-pro","displayName":"Gemini 3.1 Pro Preview","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":18,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.5-flash","displayName":"Gemini 3.5 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.5-flash-lite","displayName":"Gemini 3.5 Flash Lite","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.6-flash","displayName":"Gemini 3.6 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.7-flash","displayName":"Gemini 3.7 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"gemini-3.8-flash","displayName":"Gemini 3.8 Flash","apiDialect":"google-generative-ai","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"google-generative-ai"}}, + {"providerId":"opencode","modelId":"glm-4.6","displayName":"GLM-4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-4.7","displayName":"GLM-4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-4.7-free","displayName":"GLM-4.7 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5-free","displayName":"GLM-5 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"glm-5.3-flash","displayName":"GLM-5.3-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"gpt-5","displayName":"GPT-5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.07,"outputUsdPerMTok":8.5,"cacheReadUsdPerMTok":0.107},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5-codex","displayName":"GPT-5 Codex","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.07,"outputUsdPerMTok":8.5,"cacheReadUsdPerMTok":0.107},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5-nano","displayName":"GPT-5 Nano","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.005},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.1","displayName":"GPT-5.1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.07,"outputUsdPerMTok":8.5,"cacheReadUsdPerMTok":0.107},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.1-codex","displayName":"GPT-5.1 Codex","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.07,"outputUsdPerMTok":8.5,"cacheReadUsdPerMTok":0.107},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.1-codex-max","displayName":"GPT-5.1 Codex Max","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.1-codex-mini","displayName":"GPT-5.1 Codex Mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.2","displayName":"GPT-5.2","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.2-codex","displayName":"GPT-5.2 Codex","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.3-codex","displayName":"GPT-5.3 Codex","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.3-codex-spark","displayName":"GPT-5.3 Codex Spark","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.4","displayName":"GPT-5.4","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.4-mini","displayName":"GPT-5.4 Mini","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.4-nano","displayName":"GPT-5.4 Nano","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.4-pro","displayName":"GPT-5.4 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"cacheReadUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.5","displayName":"GPT-5.5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":10,"outputUsdPerMTok":45,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.5-pro","displayName":"GPT-5.5 Pro","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180,"cacheReadUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.6-luna","displayName":"GPT-5.6 Luna","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.6-sol","displayName":"GPT-5.6 Sol (50% Off)","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":4,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-5.6-terra","displayName":"GPT-5.6 Terra","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":3.125,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"gpt-6-astra","displayName":"GPT-6 Astra","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5,"tiers":[{"inputTokensAbove":272000,"inputUsdPerMTok":20,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":2,"cacheWriteUsdPerMTok":25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"grok-4.5","displayName":"Grok 4.5","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.3,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.6}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"grok-4.6","displayName":"Grok 4.6","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"grok-build-0.1","displayName":"Grok Build 0.1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"grok-code","displayName":"Grok Code Fast 1","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"hy3-free","displayName":"Hy3 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":190000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"hy3-preview-free","displayName":"Hy3 preview Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2","displayName":"Kimi K2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2-thinking","displayName":"Kimi K2 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.08},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2.5-free","displayName":"Kimi K2.5 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":null,"xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"laguna-s-2.1-free","displayName":"Laguna S 2.1 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"ling-2.6-flash-free","displayName":"Ling 2.6 Flash Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262100,"maxOutputTokens":32800,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"ling-3.0-flash-fin-free","displayName":"Ling 3.0 Flash Fin Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"ling-3.0-flash-free","displayName":"Ling-3.0-flash Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"ling-3.0-tiny-free","displayName":"Ling-3.0-tiny Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"longcat-2.0-free","displayName":"LongCat-2.0 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"mimo-v2-flash-free","displayName":"MiMo V2 Flash Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"mimo-v2-omni-free","displayName":"MiMo V2 Omni Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"mimo-v2-pro-free","displayName":"MiMo V2 Pro Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"mimo-v2.5-free","displayName":"MiMo V2.5 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m2.1","displayName":"MiniMax-M2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m2.1-free","displayName":"MiniMax-M2.1 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m2.5-free","displayName":"MiniMax-M2.5 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":512000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"minimax-m3-free","displayName":"MiniMax-M3 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"muse-spark-1.2","displayName":"Muse Spark 1.2","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":4.25,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"muse-spark-1.2-contributor-free","displayName":"Muse Spark 1.2 Free","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"muse-spark-1.3","displayName":"Muse Spark 1.3","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":4.25,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"muse-spark-1.3-contributor-free","displayName":"Muse Spark 1.3 Free","apiDialect":"openai-responses","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-responses","supportsDeveloperRole":true,"supportsStrictTools":true,"supportsGrammarTools":true,"supportsMaxOutputTokens":true}}, + {"providerId":"opencode","modelId":"nemotron-3-super-free","displayName":"Nemotron 3 Super Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"nemotron-3-ultra-free","displayName":"Nemotron 3 Ultra Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"nemotron-3.5-lightning-free","displayName":"Nemotron 3.5 Lightning Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"north-mini-code-free","displayName":"North Mini Code Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":256000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"qwen3-coder","displayName":"Qwen3 Coder","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.45,"outputUsdPerMTok":1.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"qwen3.5-plus","displayName":"Qwen3.5 Plus","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"qwen3.6-plus","displayName":"Qwen3.6 Plus","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05,"cacheWriteUsdPerMTok":0.625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"qwen3.6-plus-free","displayName":"Qwen3.6 Plus Free","apiDialect":"anthropic-messages","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"anthropic-messages","supportsCacheControlOnTools":true,"supportsTemperature":true,"supportsStrictTools":true}}, + {"providerId":"opencode","modelId":"ring-2.6-1t-free","displayName":"Ring 2.6 1T Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262000,"maxOutputTokens":66000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"trinity-large-preview-free","displayName":"Trinity Large Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"opencode","modelId":"x-preview-f-free","displayName":"Ox Alpha Free (Unlimited)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://opencode.ai/zen/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/qwen-token-plan-cn.generated.ts b/packages/ai/src/catalog.generated/qwen-token-plan-cn.generated.ts new file mode 100644 index 0000000..c721202 --- /dev/null +++ b/packages/ai/src/catalog.generated/qwen-token-plan-cn.generated.ts @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"qwen-token-plan-cn","modelId":"deepseek-v3.2","displayName":"DeepSeek V3.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":196608,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.6-flash","displayName":"Qwen3.6 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.6-plus","displayName":"Qwen3.6 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.7-max","displayName":"Qwen3.7 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.7-plus","displayName":"Qwen3.7 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.8-flash","displayName":"Qwen3.8 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.8-max","displayName":"Qwen3.8 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-cn","modelId":"qwen3.8-max-preview","displayName":"Qwen3.8 Max Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, +]; diff --git a/packages/ai/src/catalog.generated/qwen-token-plan-individual.generated.ts b/packages/ai/src/catalog.generated/qwen-token-plan-individual.generated.ts new file mode 100644 index 0000000..e489771 --- /dev/null +++ b/packages/ai/src/catalog.generated/qwen-token-plan-individual.generated.ts @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"qwen-token-plan-individual","modelId":"deepseek-v3.2","displayName":"DeepSeek V3.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":196608,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.6-flash","displayName":"Qwen3.6 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.6-plus","displayName":"Qwen3.6 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.7-max","displayName":"Qwen3.7 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.7-plus","displayName":"Qwen3.7 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.8-flash","displayName":"Qwen3.8 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.8-max","displayName":"Qwen3.8 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan-individual","modelId":"qwen3.8-max-preview","displayName":"Qwen3.8 Max Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://coding-intl.dashscope.aliyuncs.com/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, +]; diff --git a/packages/ai/src/catalog.generated/qwen-token-plan.generated.ts b/packages/ai/src/catalog.generated/qwen-token-plan.generated.ts new file mode 100644 index 0000000..3d81f61 --- /dev/null +++ b/packages/ai/src/catalog.generated/qwen-token-plan.generated.ts @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"qwen-token-plan","modelId":"deepseek-v3.2","displayName":"DeepSeek V3.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":196608,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.6-flash","displayName":"Qwen3.6 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.6-plus","displayName":"Qwen3.6 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.7-max","displayName":"Qwen3.7 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.7-plus","displayName":"Qwen3.7 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.8-flash","displayName":"Qwen3.8 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.8-max","displayName":"Qwen3.8 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, + {"providerId":"qwen-token-plan","modelId":"qwen3.8-max-preview","displayName":"Qwen3.8 Max Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"qwen"}}, +]; diff --git a/packages/ai/src/catalog.generated/together.generated.ts b/packages/ai/src/catalog.generated/together.generated.ts new file mode 100644 index 0000000..7f94d49 --- /dev/null +++ b/packages/ai/src/catalog.generated/together.generated.ts @@ -0,0 +1,41 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"together","modelId":"deepseek-ai/DeepSeek-V3","displayName":"DeepSeek-V3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"deepseek-ai/DeepSeek-V3-1","displayName":"DeepSeek V3.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":1.7},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"deepseek-ai/DeepSeek-V4-Flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"deepseek-ai/DeepSeek-V4-Pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":512000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":1.74,"outputUsdPerMTok":3.48,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"deepseek-ai/DeepSeek-V4-Pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":1.32,"outputUsdPerMTok":3.96,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"essentialai/Rnj-1-Instruct","displayName":"Rnj-1 Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"google/gemma-4-31B-it","displayName":"Gemma 4 31B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.39,"outputUsdPerMTok":0.97},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"meta-llama/Llama-3.3-70B-Instruct-Turbo","displayName":"Llama 3.3 70B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.04,"outputUsdPerMTok":1.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"MiniMaxAI/MiniMax-M2.5","displayName":"MiniMax-M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"MiniMaxAI/MiniMax-M2.7","displayName":"MiniMax-M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"MiniMaxAI/MiniMax-M3","displayName":"MiniMax-M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":524288,"maxOutputTokens":250000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"moonshotai/Kimi-K2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":2.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"moonshotai/Kimi-K2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"moonshotai/Kimi-K2.7-Code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.19},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"moonshotai/Kimi-K3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"nvidia/nemotron-3-ultra-550b-a55b","displayName":"Nemotron 3 Ultra 550B A55B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":512300,"maxOutputTokens":512300,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3.6,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"openai/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen2.5-7B-Instruct-Turbo","displayName":"Qwen 2.5 7B Instruct Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":false,"contextWindow":32768,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3-235B-A22B-Instruct-2507-tput","displayName":"Qwen3 235B A22B Instruct 2507 FP8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3-Coder-480B-A35B-Instruct-FP8","displayName":"Qwen3 Coder 480B A35B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3-Coder-Next-FP8","displayName":"Qwen3 Coder Next FP8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3.5-397B-A17B","displayName":"Qwen3.5 397B A17B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":130000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3.6,"cacheReadUsdPerMTok":0.35},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3.5-9B","displayName":"Qwen3.5 9B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.17,"outputUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3.6-Plus","displayName":"Qwen3.6 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"Qwen/Qwen3.7-Max","displayName":"Qwen3.7 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"thinkingmachines/Inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":524288,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":4.05,"cacheReadUsdPerMTok":0.17},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"zai-org/GLM-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"zai-org/GLM-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202752,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"zai-org/GLM-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":512000,"maxOutputTokens":164000,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"zai-org/GLM-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, + {"providerId":"together","modelId":"zai-org/GLM-5.3-Flash","displayName":"GLM-5.3-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048575,"maxOutputTokens":400000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.together.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"together"}}, +]; diff --git a/packages/ai/src/catalog.generated/vercel-ai-gateway.generated.ts b/packages/ai/src/catalog.generated/vercel-ai-gateway.generated.ts new file mode 100644 index 0000000..c02bba6 --- /dev/null +++ b/packages/ai/src/catalog.generated/vercel-ai-gateway.generated.ts @@ -0,0 +1,238 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen-3-14b","displayName":"Qwen3-14B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":40960,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.12,"outputUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen-3-235b","displayName":"Qwen3 235B A22B Instruct 2507","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.88},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen-3-30b","displayName":"Qwen3-30B-A3B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":40960,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.12,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen-3-32b","displayName":"Qwen 3.32B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.16,"outputUsdPerMTok":0.64},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen-3.6-max-preview","displayName":"Qwen 3.6 Max Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":240000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1.3,"outputUsdPerMTok":7.8,"cacheReadUsdPerMTok":0.26,"cacheWriteUsdPerMTok":1.625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-235b-a22b-thinking","displayName":"Qwen3 235B A22B Thinking 2507","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-coder","displayName":"Qwen3 Coder 480B A35B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-coder-30b-a3b","displayName":"Qwen 3 Coder 30B A3B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262144,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-coder-next","displayName":"Qwen3 Coder Next","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-coder-plus","displayName":"Qwen3 Coder Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-max","displayName":"Qwen3 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-max-preview","displayName":"Qwen3 Max Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-max-thinking","displayName":"Qwen 3 Max Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":256000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-next-80b-a3b-instruct","displayName":"Qwen3 Next 80B A3B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-next-80b-a3b-thinking","displayName":"Qwen3 Next 80B A3B Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":1.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-vl-instruct","displayName":"Qwen3 VL Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":131072,"maxOutputTokens":129024,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3-vl-thinking","displayName":"Qwen3 VL Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":131072,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.5-flash","displayName":"Qwen 3.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.001,"cacheWriteUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.5-plus","displayName":"Qwen 3.5 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.6-27b","displayName":"Qwen 3.6 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.6-plus","displayName":"Qwen 3.6 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":0.625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.7-flash","displayName":"Qwen 3.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":991000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.03,"outputUsdPerMTok":0.13,"cacheReadUsdPerMTok":0.006,"cacheWriteUsdPerMTok":0.038},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.7-max","displayName":"Qwen 3.7 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":991000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":7.5,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":3.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.7-plus","displayName":"Qwen 3.7 Plus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.08,"cacheWriteUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-2.4t-a95b","displayName":"Qwen3.8 2.4T A95B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":262144,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-27b","displayName":"Qwen3.8 27B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":0.625},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-flash","displayName":"Qwen 3.8 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":991000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.16,"outputUsdPerMTok":0.47,"cacheReadUsdPerMTok":0.016,"cacheWriteUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-flash-next","displayName":"Qwen 3.8 Flash Next","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.12,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-max","displayName":"Qwen 3.8 Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"alibaba/qwen3.8-max-0902","displayName":"Qwen3.8 Max 0902","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":null,"xhigh":"xhigh","max":null},"contextWindow":991000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"amazon/nova-lite","displayName":"Nova Lite","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.24,"cacheReadUsdPerMTok":0.015},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"amazon/nova-micro","displayName":"Nova Micro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.035,"outputUsdPerMTok":0.14,"cacheReadUsdPerMTok":0.00875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"amazon/nova-pro","displayName":"Nova Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":300000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.8,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-3-haiku","displayName":"Claude Haiku 3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":200000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-fable-5","displayName":"Claude Fable 5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-fable-5.1","displayName":"Claude Fable 5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":0.25,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-haiku-4.5","displayName":"Claude Haiku 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.1,"cacheWriteUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4","displayName":"Claude Opus 4","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":200000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.5,"cacheWriteUsdPerMTok":18.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4.5","displayName":"Claude Opus 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":200000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4.6","displayName":"Claude Opus 4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4.7","displayName":"Claude Opus 4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4.8","displayName":"Claude Opus 4.8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-4.8-fast","displayName":"Claude Opus 4.8 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-5","displayName":"Claude Opus 5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":25,"cacheReadUsdPerMTok":0.5,"cacheWriteUsdPerMTok":6.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-opus-5-fast","displayName":"Claude Opus 5 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-sonnet-4","displayName":"Claude Sonnet 4","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-sonnet-4.5","displayName":"Claude Sonnet 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-sonnet-4.6","displayName":"Claude Sonnet 4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3,"cacheWriteUsdPerMTok":3.75,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":6,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.6,"cacheWriteUsdPerMTok":7.5}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"anthropic/claude-sonnet-5","displayName":"Claude Sonnet 5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"arcee-ai/trinity-large-thinking","displayName":"Trinity Large Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":262100,"maxOutputTokens":80000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":0.8999999999999999},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"bytedance/seed-1.6","displayName":"Seed 1.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"bytedance/seed-1.8","displayName":"Seed 1.8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"cohere/command-a","displayName":"Command A","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":8000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-r1","displayName":"DeepSeek-R1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.35,"outputUsdPerMTok":5.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v3.1","displayName":"DeepSeek-V3.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":163840,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":0.95,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v3.1-terminus","displayName":"DeepSeek V3.1 Terminus","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.27,"outputUsdPerMTok":1,"cacheReadUsdPerMTok":0.135},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v3.2-thinking","displayName":"DeepSeek V3.2 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":128000,"maxOutputTokens":8000,"cost":{"inputUsdPerMTok":0.62,"outputUsdPerMTok":1.85},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v4-flash","displayName":"DeepSeek V4 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.13,"outputUsdPerMTok":0.26,"cacheReadUsdPerMTok":0.028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v4-flash-0731","displayName":"DeepSeek V4 Flash 0731","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.076,"outputUsdPerMTok":0.153,"cacheReadUsdPerMTok":0.014},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v4-flash-vision-exp","displayName":"DeepSeek V4 Flash Vision Exp","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.66,"cacheReadUsdPerMTok":0.007},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v4-pro","displayName":"DeepSeek V4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.66,"outputUsdPerMTok":1.98,"cacheReadUsdPerMTok":0.022},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"deepseek/deepseek-v4-pro-0813","displayName":"DeepSeek V4 Pro 0813","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":384000,"cost":{"inputUsdPerMTok":0.66,"outputUsdPerMTok":1.98,"cacheReadUsdPerMTok":0.066},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-2.5-flash","displayName":"Gemini 2.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-2.5-flash-lite","displayName":"Gemini 2.5 Flash Lite","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-2.5-pro","displayName":"Gemini 2.5 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":1048576,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3-flash","displayName":"Gemini 3 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.1-flash-lite","displayName":"Gemini 3.1 Flash Lite","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.1-pro-preview","displayName":"Gemini 3.1 Pro Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.5-flash","displayName":"Gemini 3.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.5-flash-lite","displayName":"Gemini 3.5 Flash Lite","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.6-flash","displayName":"Gemini 3.6 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.7-flash","displayName":"Gemini 3.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemini-3.8-flash","displayName":"Gemini 3.8 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":3.75,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemma-4-26b-a4b-it","displayName":"Gemma 4 26B A4B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.015},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"google/gemma-4-31b-it","displayName":"Gemma 4 31B IT","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inception/mercury-2","displayName":"Mercury 2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":0.75,"cacheReadUsdPerMTok":0.024999999999999998},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inception/mercury-coder-small","displayName":"Mercury Coder Small Beta","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":32000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inclusionai/ling-3.0-flash","displayName":"Ling 3.0 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.18,"cacheReadUsdPerMTok":0.012},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inclusionai/ling-3.0-flash-fin","displayName":"Ling 3.0 Flash Fin","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inclusionai/ling-3.0-flash-fin-free","displayName":"Ling 3.0 Flash Fin (Free)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inclusionai/ling-3.0-flash-sante","displayName":"Ling 3.0 Flash Sante","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"inclusionai/ling-3.0-flash-sante-free","displayName":"Ling 3.0 Flash Sante (Free)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":32000,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"kwaipilot/kat-coder-air-v2.5","displayName":"Kat Coder Air V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":80000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"kwaipilot/kat-coder-pro-v2","displayName":"Kat Coder Pro V2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"kwaipilot/kat-coder-pro-v2.5","displayName":"Kat Coder Pro V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":80000,"cost":{"inputUsdPerMTok":0.74,"outputUsdPerMTok":2.96,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/llama-3.1-70b","displayName":"Llama 3.1 70B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.72,"outputUsdPerMTok":0.72},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/llama-3.1-8b","displayName":"Llama 3.1 8B Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.22,"outputUsdPerMTok":0.22},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/llama-3.3-70b","displayName":"Llama-3.3-70B-Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/llama-4-maverick","displayName":"Llama-4-Maverick-17B-128E-Instruct-FP8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/llama-4-scout","displayName":"Llama-4-Scout-17B-16E-Instruct-FP8","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-glimmer-30b","displayName":"Muse Glimmer 30B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.35,"outputUsdPerMTok":1.5,"cacheReadUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-spark-1.1","displayName":"Muse Spark 1.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":4.25,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-spark-1.2","displayName":"Muse Spark 1.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":4.25,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-spark-1.2-contributor","displayName":"Muse Spark 1.2 Contributor","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.002},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-spark-1.3","displayName":"Muse Spark 1.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max"},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":4.25,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"meta/muse-spark-1.3-contributor","displayName":"Muse Spark 1.3 Contributor","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.002},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2","displayName":"MiniMax M2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":205000,"maxOutputTokens":205000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.1","displayName":"MiniMax M2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.1-lightning","displayName":"MiniMax M2.1 Lightning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.5","displayName":"MiniMax M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.5-highspeed","displayName":"MiniMax M2.5 High Speed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.7","displayName":"Minimax M2.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.7-free","displayName":"Minimax M2.7 (Free)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":196608,"maxOutputTokens":196608,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m2.7-highspeed","displayName":"MiniMax M2.7 High Speed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":204800,"maxOutputTokens":131100,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.06,"cacheWriteUsdPerMTok":0.375},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m3","displayName":"MiniMax M3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":512000,"maxOutputTokens":512000,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.06},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"minimax/minimax-m3-free","displayName":"MiniMax M3 (Free)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/codestral","displayName":"Codestral (latest)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/devstral-2","displayName":"Devstral 2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/devstral-small-2","displayName":"Devstral Small 2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/ministral-3b","displayName":"Ministral 3B (latest)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.04,"outputUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/ministral-8b","displayName":"Ministral 8B (latest)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/mistral-medium","displayName":"Mistral Medium 3.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/mistral-medium-3.5","displayName":"Mistral Medium Latest","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/mistral-nemo","displayName":"Mistral Nemo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/mistral-small","displayName":"Mistral Small (latest)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":32000,"maxOutputTokens":4000,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"mistral/pixtral-12b","displayName":"Pixtral 12B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2","displayName":"Kimi K2 Instruct","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":false,"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.57,"outputUsdPerMTok":2.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2-thinking","displayName":"Kimi K2 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":216144,"maxOutputTokens":216144,"cost":{"inputUsdPerMTok":0.47,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.141},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2.5","displayName":"Kimi K2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262114,"maxOutputTokens":262114,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":3,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2.6","displayName":"Kimi K2.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262000,"maxOutputTokens":262000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2.7-code","displayName":"Kimi K2.7 Code","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k2.7-code-highspeed","displayName":"Kimi K2.7 Code High Speed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":262144,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":1.9,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.38},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k3","displayName":"Kimi K3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":3,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"moonshotai/kimi-k3-fast","displayName":"Kimi K3 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":4.5,"outputUsdPerMTok":22.5,"cacheReadUsdPerMTok":0.45},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"nvidia/nemotron-3-ultra-550b-a55b","displayName":"Nemotron 3 Ultra","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":65000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"nvidia/nemotron-3.5-lightning","displayName":"Nemotron 3.5 Lightning 30B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":"1024","low":"2048","medium":"8192","high":"16384","xhigh":"16384","max":"16384"},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"nvidia/nemotron-nano-12b-v2-vl","displayName":"Nvidia Nemotron Nano 12B V2 VL","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"nvidia/nemotron-nano-9b-v2","displayName":"Nvidia Nemotron Nano 9B V2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.23},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4-turbo","displayName":"GPT-4 Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":4096,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":30},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1","displayName":"GPT-4.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1-fast","displayName":"GPT-4.1 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":3.5,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1-mini","displayName":"GPT-4.1 mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":1.6,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1-mini-fast","displayName":"GPT-4.1 mini (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.7,"outputUsdPerMTok":2.8,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1-nano","displayName":"GPT-4.1 nano","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4.1-nano-fast","displayName":"GPT-4.1 nano (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1047576,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.8,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4o","displayName":"GPT-4o","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4o-fast","displayName":"GPT-4o (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":4.25,"outputUsdPerMTok":17,"cacheReadUsdPerMTok":2.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4o-mini","displayName":"GPT-4o mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-4o-mini-fast","displayName":"GPT-4o mini (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":128000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":1,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5","displayName":"GPT-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-codex","displayName":"GPT-5-Codex","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-fast","displayName":"GPT-5 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-mini","displayName":"GPT-5 Mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.025},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-mini-fast","displayName":"GPT-5 mini (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.45,"outputUsdPerMTok":3.6,"cacheReadUsdPerMTok":0.045},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-nano","displayName":"GPT-5 Nano","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.005},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5-pro","displayName":"GPT-5 pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":272000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":120},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.1-codex","displayName":"GPT-5.1-Codex","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.1-codex-max","displayName":"GPT 5.1 Codex Max","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.1-codex-mini","displayName":"GPT-5.1 Codex mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.25,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.1-thinking","displayName":"GPT 5.1 Thinking","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.125},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.1-thinking-fast","displayName":"GPT 5.1 Thinking (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.2","displayName":"GPT-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.2-codex","displayName":"GPT-5.2-Codex","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.2-fast","displayName":"GPT 5.2 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3.5,"outputUsdPerMTok":28,"cacheReadUsdPerMTok":0.35},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.2-pro","displayName":"GPT 5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":21,"outputUsdPerMTok":168},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.3-codex","displayName":"GPT 5.3 Codex","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.75,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.175},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.3-codex-fast","displayName":"GPT 5.3 Codex (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":3.5,"outputUsdPerMTok":28,"cacheReadUsdPerMTok":0.35},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4","displayName":"GPT 5.4","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.5,"outputUsdPerMTok":15,"cacheReadUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4-fast","displayName":"GPT 5.4 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4-mini","displayName":"GPT 5.4 Mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.75,"outputUsdPerMTok":4.5,"cacheReadUsdPerMTok":0.075},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4-mini-fast","displayName":"GPT 5.4 Mini (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.5,"outputUsdPerMTok":9,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4-nano","displayName":"GPT 5.4 Nano","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":400000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.25,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.4-pro","displayName":"GPT 5.4 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.5","displayName":"GPT 5.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.5-fast","displayName":"GPT 5.5 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null,"off":"none"},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":12.5,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":1.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.5-pro","displayName":"GPT 5.5 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":30,"outputUsdPerMTok":180},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-luna","displayName":"GPT 5.6 Luna","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.02,"cacheWriteUsdPerMTok":0.25},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-luna-fast","displayName":"GPT 5.6 Luna (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.4,"outputUsdPerMTok":2.4,"cacheReadUsdPerMTok":0.04,"cacheWriteUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-sol","displayName":"GPT 5.6 Sol","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":10,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-sol-fast","displayName":"GPT 5.6 Sol (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":20,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-terra","displayName":"GPT 5.6 Terra","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":2.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-5.6-terra-fast","displayName":"GPT 5.6 Terra (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":4,"outputUsdPerMTok":24,"cacheReadUsdPerMTok":0.4,"cacheWriteUsdPerMTok":5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-6-astra","displayName":"GPT-6 Astra","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":10,"outputUsdPerMTok":50,"cacheReadUsdPerMTok":1,"cacheWriteUsdPerMTok":12.5,"tiers":[{"inputTokensAbove":272001,"inputUsdPerMTok":20,"outputUsdPerMTok":75,"cacheReadUsdPerMTok":2,"cacheWriteUsdPerMTok":25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-6-astra-fast","displayName":"GPT-6 Astra (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1050000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":20,"outputUsdPerMTok":100,"cacheReadUsdPerMTok":2,"cacheWriteUsdPerMTok":25,"tiers":[{"inputTokensAbove":272001,"inputUsdPerMTok":40,"outputUsdPerMTok":150,"cacheReadUsdPerMTok":4,"cacheWriteUsdPerMTok":25}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-oss-120b","displayName":"GPT OSS 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-oss-20b","displayName":"GPT OSS 20B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":131072,"maxOutputTokens":8192,"cost":{"inputUsdPerMTok":0.05,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-oss-safeguard-120b","displayName":"GPT OSS Safeguard 120B","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.6},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/gpt-oss-safeguard-20b","displayName":"gpt-oss-safeguard-20b","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":128000,"maxOutputTokens":16000,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o1","displayName":"o1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":15,"outputUsdPerMTok":60,"cacheReadUsdPerMTok":7.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o3","displayName":"o3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o3-fast","displayName":"o3 (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":3.5,"outputUsdPerMTok":14,"cacheReadUsdPerMTok":0.875},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o3-mini","displayName":"o3-mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.55},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o3-pro","displayName":"o3 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":20,"outputUsdPerMTok":80},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o4-mini","displayName":"o4-mini","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":1.1,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.275},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"openai/o4-mini-fast","displayName":"o4-mini (Fast)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":200000,"maxOutputTokens":100000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":8,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"perplexity/sonar","displayName":"Sonar","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":127000,"maxOutputTokens":8000,"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"perplexity/sonar-pro","displayName":"Sonar Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":200000,"maxOutputTokens":8000,"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"poolside/laguna-s-2.1","displayName":"Laguna S 2.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.1,"outputUsdPerMTok":0.2,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"poolside/laguna-s-2.1-free","displayName":"Laguna S 2.1 Free","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":256000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"sakana/fugu-ultra","displayName":"Fugu Ultra","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":5,"outputUsdPerMTok":30,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"sakana/namazu","displayName":"Sakana Namazu","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.95,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.15},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.1-fast-non-reasoning","displayName":"Grok 4.1 Fast Non-Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.1-fast-reasoning","displayName":"Grok 4.1 Fast Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.05},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-multi-agent","displayName":"Grok 4.20 Multi-Agent","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-multi-agent-beta","displayName":"Grok 4.20 Multi Agent Beta","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-non-reasoning","displayName":"Grok 4.20 Non-Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-non-reasoning-beta","displayName":"Grok 4.20 Beta Non-Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":false,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-reasoning","displayName":"Grok 4.20 Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.20-reasoning-beta","displayName":"Grok 4.20 Beta Reasoning","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"contextWindow":2000000,"maxOutputTokens":2000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.3","displayName":"Grok 4.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.5","displayName":"Grok 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.3},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-4.6","displayName":"Grok 4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.5},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"spacexai/grok-build-0.1","displayName":"Grok Build 0.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"stepfun/step-3.5-flash","displayName":"StepFun 3.5 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":262114,"maxOutputTokens":262114,"cost":{"inputUsdPerMTok":0.09,"outputUsdPerMTok":0.3,"cacheReadUsdPerMTok":0.02},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"stepfun/step-3.7-flash","displayName":"Step 3.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.15,"cacheReadUsdPerMTok":0.04},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"tencent/hy3","displayName":"Hy3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":262144,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.58,"cacheReadUsdPerMTok":0.035},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"tencent/hy4-preview","displayName":"Tencent Hy4 Preview","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":1024000,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":0.834,"outputUsdPerMTok":2.501,"cacheReadUsdPerMTok":0.042},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"thinkingmachines/inkling","displayName":"Inkling","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":4.05,"cacheReadUsdPerMTok":0.17},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"thinkingmachines/inkling-small","displayName":"Inkling Small","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":"minimal","low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":"max","off":"none"},"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":0.5,"outputUsdPerMTok":1.2,"cacheReadUsdPerMTok":0.1},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"xiaomi/mimo-v2.5","displayName":"MiMo M2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1050000,"maxOutputTokens":131100,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"xiaomi/mimo-v2.5-pro","displayName":"MiMo V2.5 Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1050000,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.0036},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"xiaomi/mimo-v2.5-pro-ultraspeed","displayName":"MiMo V2.5 Pro UltraSpeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.305,"outputUsdPerMTok":2.61,"cacheReadUsdPerMTok":0.0108},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.5","displayName":"GLM 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":128000,"maxOutputTokens":96000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.5-air","displayName":"GLM 4.5 Air","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":128000,"maxOutputTokens":96000,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.1,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.5v","displayName":"GLM 4.5V","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":66000,"maxOutputTokens":16000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":1.8,"cacheReadUsdPerMTok":0.11},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.6","displayName":"GLM 4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":96000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.7","displayName":"GLM 4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":120000,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.12},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.7-flash","displayName":"GLM 4.7 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.4},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-4.7-flashx","displayName":"GLM 4.7 FlashX","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.06,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":131100,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5-turbo","displayName":"GLM 5 Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":131100,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.1","displayName":"GLM 5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":202800,"maxOutputTokens":64000,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.2","displayName":"GLM 5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":0.8,"outputUsdPerMTok":2.55,"cacheReadUsdPerMTok":0.16},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.2-fast","displayName":"GLM 5.2 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":"xhigh","max":null},"contextWindow":1000000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":2.1,"outputUsdPerMTok":6.6,"cacheReadUsdPerMTok":0.21},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.3","displayName":"GLM 5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":1000000,"cost":{"inputUsdPerMTok":0.7,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.3-fast","displayName":"GLM 5.3 Fast","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1048576,"maxOutputTokens":262144,"cost":{"inputUsdPerMTok":2.1,"outputUsdPerMTok":6.6,"cacheReadUsdPerMTok":0.21},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.3-flash","displayName":"GLM 5.3 Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131000,"cost":{"inputUsdPerMTok":0.15,"outputUsdPerMTok":0.5,"cacheReadUsdPerMTok":0.03},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5.3-promo-50","displayName":"GLM 5.3 (50% off)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"contextWindow":1048576,"maxOutputTokens":1048576,"cost":{"inputUsdPerMTok":0.7,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.13},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"vercel-ai-gateway","modelId":"zai/glm-5v-turbo","displayName":"GLM 5V Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":128000,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.24},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://ai-gateway.vercel.sh/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/xai.generated.ts b/packages/ai/src/catalog.generated/xai.generated.ts new file mode 100644 index 0000000..a7be43c --- /dev/null +++ b/packages/ai/src/catalog.generated/xai.generated.ts @@ -0,0 +1,15 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"xai","modelId":"grok-4.20-0309-non-reasoning","displayName":"Grok 4.20 (Non-Reasoning)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":false,"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xai","modelId":"grok-4.20-0309-reasoning","displayName":"Grok 4.20 (Reasoning)","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xai","modelId":"grok-4.3","displayName":"Grok 4.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null,"off":"none"},"contextWindow":1000000,"maxOutputTokens":30000,"cost":{"inputUsdPerMTok":1.25,"outputUsdPerMTok":2.5,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2.5,"outputUsdPerMTok":5,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xai","modelId":"grok-4.5","displayName":"Grok 4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":null,"max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.3,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":0.6}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xai","modelId":"grok-4.6","displayName":"Grok 4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":"medium","high":"high","xhigh":"xhigh","max":null},"contextWindow":500000,"maxOutputTokens":500000,"cost":{"inputUsdPerMTok":2,"outputUsdPerMTok":6,"cacheReadUsdPerMTok":0.5,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":4,"outputUsdPerMTok":12,"cacheReadUsdPerMTok":1}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xai","modelId":"grok-build-0.1","displayName":"Grok Build 0.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"contextWindow":256000,"maxOutputTokens":256000,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":2,"cacheReadUsdPerMTok":0.2,"tiers":[{"inputTokensAbove":200000,"inputUsdPerMTok":2,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.4}]},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.x.ai/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/xiaomi-token-plan-ams.generated.ts b/packages/ai/src/catalog.generated/xiaomi-token-plan-ams.generated.ts new file mode 100644 index 0000000..6b24a4c --- /dev/null +++ b/packages/ai/src/catalog.generated/xiaomi-token-plan-ams.generated.ts @@ -0,0 +1,12 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"xiaomi-token-plan-ams","modelId":"mimo-v2-pro","displayName":"MiMo-V2-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-ams.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-ams","modelId":"mimo-v2.5","displayName":"MiMo-V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-ams.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-ams","modelId":"mimo-v2.5-pro","displayName":"MiMo-V2.5-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-ams.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/xiaomi-token-plan-cn.generated.ts b/packages/ai/src/catalog.generated/xiaomi-token-plan-cn.generated.ts new file mode 100644 index 0000000..a1c4bc2 --- /dev/null +++ b/packages/ai/src/catalog.generated/xiaomi-token-plan-cn.generated.ts @@ -0,0 +1,12 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"xiaomi-token-plan-cn","modelId":"mimo-v2-pro","displayName":"MiMo-V2-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-cn.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-cn","modelId":"mimo-v2.5","displayName":"MiMo-V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-cn.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-cn","modelId":"mimo-v2.5-pro","displayName":"MiMo-V2.5-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-cn.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/xiaomi-token-plan-sgp.generated.ts b/packages/ai/src/catalog.generated/xiaomi-token-plan-sgp.generated.ts new file mode 100644 index 0000000..6bdfe60 --- /dev/null +++ b/packages/ai/src/catalog.generated/xiaomi-token-plan-sgp.generated.ts @@ -0,0 +1,12 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"xiaomi-token-plan-sgp","modelId":"mimo-v2-pro","displayName":"MiMo-V2-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-sgp.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-sgp","modelId":"mimo-v2.5","displayName":"MiMo-V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-sgp.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi-token-plan-sgp","modelId":"mimo-v2.5-pro","displayName":"MiMo-V2.5-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://token-plan-sgp.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/xiaomi.generated.ts b/packages/ai/src/catalog.generated/xiaomi.generated.ts new file mode 100644 index 0000000..e503bdd --- /dev/null +++ b/packages/ai/src/catalog.generated/xiaomi.generated.ts @@ -0,0 +1,15 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"xiaomi","modelId":"mimo-v2-flash","displayName":"MiMo-V2-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":65536,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi","modelId":"mimo-v2-omni","displayName":"MiMo-V2-Omni","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":262144,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi","modelId":"mimo-v2-pro","displayName":"MiMo-V2-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.0036},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"deprecated","reason":"Deprecated by the source catalog"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi","modelId":"mimo-v2.5","displayName":"MiMo-V2.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.14,"outputUsdPerMTok":0.28,"cacheReadUsdPerMTok":0.0028},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi","modelId":"mimo-v2.5-pro","displayName":"MiMo-V2.5-Pro","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.435,"outputUsdPerMTok":0.87,"cacheReadUsdPerMTok":0.0036},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, + {"providerId":"xiaomi","modelId":"mimo-v2.5-pro-ultraspeed","displayName":"MiMo-V2.5-Pro-UltraSpeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":1048576,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.305,"outputUsdPerMTok":2.61,"cacheReadUsdPerMTok":0.0108},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.xiaomimimo.com/v1"},"availability":{"status":"preview","reason":"Source catalog status: beta"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false}}, +]; diff --git a/packages/ai/src/catalog.generated/zai-coding-cn.generated.ts b/packages/ai/src/catalog.generated/zai-coding-cn.generated.ts new file mode 100644 index 0000000..e542fa8 --- /dev/null +++ b/packages/ai/src/catalog.generated/zai-coding-cn.generated.ts @@ -0,0 +1,19 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"zai-coding-cn","modelId":"glm-4.6v","displayName":"GLM-4.6V","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-4.7","displayName":"GLM-4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5-turbo","displayName":"GLM-5-Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.2-highspeed","displayName":"GLM-5.2 Highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.3-flash","displayName":"GLM-5.3-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5.3-highspeed","displayName":"GLM-5.3 Highspeed","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai-coding-cn","modelId":"glm-5v-turbo","displayName":"GLM-5V-Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://open.bigmodel.cn/api/coding/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, +]; diff --git a/packages/ai/src/catalog.generated/zai.generated.ts b/packages/ai/src/catalog.generated/zai.generated.ts new file mode 100644 index 0000000..fa10e5b --- /dev/null +++ b/packages/ai/src/catalog.generated/zai.generated.ts @@ -0,0 +1,25 @@ +// SPDX-FileCopyrightText: 2025 models.dev contributors +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: MIT +// @generated by packages/ai/scripts/generate-catalog.ts; do not edit. + +import type { ModelInfo } from "../model.ts"; + +export const MODELS: readonly ModelInfo[] = [ + {"providerId":"zai","modelId":"glm-4.5","displayName":"GLM-4.5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.5-air","displayName":"GLM-4.5-Air","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0.2,"outputUsdPerMTok":1.1,"cacheReadUsdPerMTok":0.03,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.5-flash","displayName":"GLM-4.5-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":131072,"maxOutputTokens":98304,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.5v","displayName":"GLM-4.5V","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":64000,"maxOutputTokens":16384,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":1.8},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.6","displayName":"GLM-4.6","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.6v","displayName":"GLM-4.6V","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":128000,"maxOutputTokens":32768,"cost":{"inputUsdPerMTok":0.3,"outputUsdPerMTok":0.9},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.7","displayName":"GLM-4.7","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.6,"outputUsdPerMTok":2.2,"cacheReadUsdPerMTok":0.11,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.7-flash","displayName":"GLM-4.7-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0,"outputUsdPerMTok":0,"cacheReadUsdPerMTok":0,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-4.7-flashx","displayName":"GLM-4.7-FlashX","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.07,"outputUsdPerMTok":0.4,"cacheReadUsdPerMTok":0.01,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5","displayName":"GLM-5","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":204800,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1,"outputUsdPerMTok":3.2,"cacheReadUsdPerMTok":0.2,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5-turbo","displayName":"GLM-5-Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.24,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5.1","displayName":"GLM-5.1","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5.2","displayName":"GLM-5.2","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":null,"medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5.3","displayName":"GLM-5.3","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":false},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.4,"outputUsdPerMTok":4.4,"cacheReadUsdPerMTok":0.26,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5.3-flash","displayName":"GLM-5.3-Flash","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":true,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"minimal":null,"low":"low","medium":null,"high":"high","xhigh":null,"max":"max"},"contextWindow":1000000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":0.075,"outputUsdPerMTok":0.25,"cacheReadUsdPerMTok":0.015,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, + {"providerId":"zai","modelId":"glm-5v-turbo","displayName":"GLM-5V-Turbo","apiDialect":"openai-chat","capabilities":{"toolUse":true,"structuredOutput":false,"imageInput":true},"reasoning":true,"thinkingLevelMap":{"off":"disabled","minimal":null,"low":null,"medium":null,"high":"enabled"},"contextWindow":200000,"maxOutputTokens":131072,"cost":{"inputUsdPerMTok":1.2,"outputUsdPerMTok":4,"cacheReadUsdPerMTok":0.24,"cacheWriteUsdPerMTok":0},"cache":{"supported":true,"defaultRetention":"short","supportedRetentions":["none","short"]},"endpoint":{"type":"fixed","baseUrl":"https://api.z.ai/api/paas/v4"},"availability":{"status":"available"},"compatibility":{"dialect":"openai-chat","supportsStore":false,"supportsDeveloperRole":false,"supportsReasoningEffort":false,"supportsUsageInStreaming":true,"maxTokensField":"max_tokens","supportsStrictTools":false,"supportsLongCacheRetention":false,"thinkingFormat":"zai"}}, +]; diff --git a/packages/ai/src/catalog.ts b/packages/ai/src/catalog.ts new file mode 100644 index 0000000..239a04c --- /dev/null +++ b/packages/ai/src/catalog.ts @@ -0,0 +1,46 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { ModelInfo } from "./model.ts"; +import { + BUILTIN_CATALOG_PROVIDERS, + GENERATED_CATALOG_PROVENANCE, + STATIC_MODEL_CATALOG, +} from "./catalog.generated.ts"; + +export { + ModelCatalogValidationError, + validateModelCatalog, +} from "./catalog-validation.ts"; + +export interface BuiltinCatalogProvider { + readonly id: string; + readonly displayName: string; + readonly catalogKind: "static" | "dynamic" | "configured"; + readonly regionFamily?: string; + readonly region?: string; +} + +export interface CatalogProvenance { + readonly generatedAt: string; + readonly sources: readonly { + readonly name: string; + readonly location: string; + readonly retrievedAt: string; + readonly sha256?: string; + readonly revision?: string; + readonly license: string; + }[]; +} + +/** Synchronous static lookup with no network or credential access. */ +export function getStaticModelCatalog(providerId: string): readonly ModelInfo[] { + const catalog: Readonly> = STATIC_MODEL_CATALOG; + return catalog[providerId] ?? []; +} + +export function listBuiltinCatalogProviders(): readonly BuiltinCatalogProvider[] { + return BUILTIN_CATALOG_PROVIDERS; +} + +export { GENERATED_CATALOG_PROVENANCE, STATIC_MODEL_CATALOG }; diff --git a/packages/ai/src/cerebras.ts b/packages/ai/src/cerebras.ts new file mode 100644 index 0000000..f6a8866 --- /dev/null +++ b/packages/ai/src/cerebras.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const CEREBRAS_PROVIDER_ID = "cerebras"; +export const CEREBRAS_API_KEY_ENV = "CEREBRAS_API_KEY"; +export const CEREBRAS_BASE_URL = "https://api.cerebras.ai/v1"; + +export const CEREBRAS_PROVIDER_DEFINITION = { + id: CEREBRAS_PROVIDER_ID, + displayName: "Cerebras", + apiKeyDisplayName: "Cerebras API key", + environmentVariables: [CEREBRAS_API_KEY_ENV], + baseUrl: CEREBRAS_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createCerebrasProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(CEREBRAS_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/cloud-auth.ts b/packages/ai/src/cloud-auth.ts new file mode 100644 index 0000000..7322b4c --- /dev/null +++ b/packages/ai/src/cloud-auth.ts @@ -0,0 +1,415 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { DefaultAzureCredential } from "@azure/identity"; +import { GoogleAuth } from "google-auth-library"; + +import { + type AmbientAuthSource, + type ApiKeyAuthMethod, + type AuthContext, + AuthError, + type ResolvedAuth, +} from "./auth.ts"; +import type { ApiKeyCredential, ProviderEnv } from "./credentials.ts"; +import { raceWithSignal } from "./transport-safety.ts"; + +const AZURE_SCOPE = "https://cognitiveservices.azure.com/.default"; +const GOOGLE_SCOPE = "https://www.googleapis.com/auth/cloud-platform"; + +export interface AccessTokenValue { + readonly token: string; + readonly expiresOnTimestamp?: number; +} + +export interface AzureCredentialLike { + getToken( + scope: string, + options?: { readonly abortSignal?: AbortSignal }, + ): Promise; +} + +export interface GoogleAuthClientLike { + getAccessToken(): Promise; +} + +export interface GoogleAuthLike { + getClient(): Promise; + getProjectId(): Promise; +} + +export interface CloudAuthFactories { + readonly azureCredential?: () => AzureCredentialLike; + readonly googleAuth?: (options: { + readonly scopes: string; + readonly keyFilename?: string; + readonly projectId?: string; + }) => GoogleAuthLike; +} + +function required(value: string | undefined, label: string, providerId: string): string { + const result = value?.trim(); + if (!result) throw new AuthError("not_configured", providerId, `${providerId} requires ${label}`); + return result; +} + +function safeToken(value: string | null | undefined, providerId: string): string { + if (!value || /[\r\n]/.test(value)) { + throw new AuthError( + "invalid_auth", + providerId, + `${providerId} returned an invalid access token`, + ); + } + return value; +} + +function azureSettings( + context: AuthContext, + credential?: ApiKeyCredential, +): Readonly> { + const setting = (name: string) => credential?.env?.[name] ?? context.env(name); + const explicitBase = setting("AZURE_OPENAI_BASE_URL")?.trim(); + const resource = setting("AZURE_OPENAI_RESOURCE_NAME")?.trim(); + if (!explicitBase && !resource) { + throw new AuthError( + "not_configured", + "azure-openai-responses", + "Azure OpenAI requires AZURE_OPENAI_BASE_URL or AZURE_OPENAI_RESOURCE_NAME", + ); + } + return { + AZURE_OPENAI_BASE_URL: explicitBase ?? `https://${resource}.openai.azure.com/openai/v1`, + ...(setting("AZURE_OPENAI_API_VERSION") + ? { AZURE_OPENAI_API_VERSION: setting("AZURE_OPENAI_API_VERSION") as string } + : {}), + ...(setting("AZURE_OPENAI_DEPLOYMENT_NAME_MAP") + ? { + AZURE_OPENAI_DEPLOYMENT_NAME_MAP: setting("AZURE_OPENAI_DEPLOYMENT_NAME_MAP") as string, + } + : {}), + }; +} + +export function createAzureEntraSource(factories: CloudAuthFactories = {}): AmbientAuthSource { + return { + type: "ambient", + displayName: "Microsoft Entra credentials", + resolve: async ({ context, signal }) => { + signal.throwIfAborted(); + const env = azureSettings(context); + const credential = + factories.azureCredential?.() ?? + new DefaultAzureCredential({ + requiredEnvVars: [], + }); + let access: AccessTokenValue | null; + try { + access = await credential.getToken(AZURE_SCOPE, { abortSignal: signal }); + } catch (cause) { + if (signal.aborted) signal.throwIfAborted(); + throw new AuthError( + "invalid_auth", + "azure-openai-responses", + "Microsoft Entra credential acquisition failed for azure-openai-responses", + cause, + ); + } + signal.throwIfAborted(); + const token = safeToken(access?.token, "azure-openai-responses"); + return { + auth: { headers: { authorization: `Bearer ${token}` } }, + env, + source: "Microsoft Entra default credential chain", + secretValues: [token], + }; + }, + }; +} + +function vertexSettings( + context: AuthContext, + credential: ApiKeyCredential | undefined, +): { + project?: string; + location?: string; + keyFilename?: string; + baseUrl?: string; + apiVersion?: string; +} { + const setting = (name: string) => credential?.env?.[name] ?? context.env(name); + return { + ...((setting("GOOGLE_CLOUD_PROJECT") ?? setting("GCLOUD_PROJECT")) + ? { project: (setting("GOOGLE_CLOUD_PROJECT") ?? setting("GCLOUD_PROJECT")) as string } + : {}), + ...(setting("GOOGLE_CLOUD_LOCATION") + ? { location: setting("GOOGLE_CLOUD_LOCATION") as string } + : {}), + ...(setting("GOOGLE_APPLICATION_CREDENTIALS") + ? { keyFilename: setting("GOOGLE_APPLICATION_CREDENTIALS") as string } + : {}), + ...(setting("GOOGLE_VERTEX_BASE_URL") + ? { baseUrl: setting("GOOGLE_VERTEX_BASE_URL") as string } + : {}), + ...(setting("GOOGLE_VERTEX_API_VERSION") + ? { apiVersion: setting("GOOGLE_VERTEX_API_VERSION") as string } + : {}), + }; +} + +async function googleAccessToken( + context: AuthContext, + credential: ApiKeyCredential | undefined, + signal: AbortSignal, + factories: CloudAuthFactories, + mode: "adc" | "service_account", +): Promise { + const settings = vertexSettings(context, credential); + const location = required(settings.location, "GOOGLE_CLOUD_LOCATION", "google-vertex"); + if (mode === "service_account") { + required(settings.keyFilename, "GOOGLE_APPLICATION_CREDENTIALS", "google-vertex"); + } + signal.throwIfAborted(); + const auth = + factories.googleAuth?.({ + scopes: GOOGLE_SCOPE, + ...(settings.keyFilename === undefined ? {} : { keyFilename: settings.keyFilename }), + ...(settings.project === undefined ? {} : { projectId: settings.project }), + }) ?? + new GoogleAuth({ + scopes: GOOGLE_SCOPE, + ...(settings.keyFilename === undefined ? {} : { keyFilename: settings.keyFilename }), + ...(settings.project === undefined ? {} : { projectId: settings.project }), + }); + try { + const [client, discoveredProject] = await raceWithSignal( + Promise.all([ + auth.getClient(), + settings.project === undefined ? auth.getProjectId() : Promise.resolve(settings.project), + ]), + signal, + ); + signal.throwIfAborted(); + const access = await raceWithSignal(client.getAccessToken(), signal); + signal.throwIfAborted(); + const token = safeToken(typeof access === "string" ? access : access?.token, "google-vertex"); + return { + auth: { headers: { authorization: `Bearer ${token}` } }, + env: { + GOOGLE_CLOUD_PROJECT: required(discoveredProject, "project ID", "google-vertex"), + GOOGLE_CLOUD_LOCATION: location, + GOOGLE_VERTEX_CREDENTIAL_TYPE: mode, + ...(settings.keyFilename === undefined + ? {} + : { GOOGLE_APPLICATION_CREDENTIALS: settings.keyFilename }), + ...(settings.baseUrl === undefined ? {} : { GOOGLE_VERTEX_BASE_URL: settings.baseUrl }), + ...(settings.apiVersion === undefined + ? {} + : { GOOGLE_VERTEX_API_VERSION: settings.apiVersion }), + }, + source: + mode === "service_account" + ? "Google service account credentials" + : "Google Application Default Credentials", + secretValues: [token], + }; + } catch (cause) { + if (signal.aborted) signal.throwIfAborted(); + if (cause instanceof AuthError) throw cause; + throw new AuthError( + "invalid_auth", + "google-vertex", + `Google ${mode === "service_account" ? "service account" : "ADC"} credential acquisition failed`, + cause, + ); + } +} + +function vertexApiKey( + context: AuthContext, + credential?: ApiKeyCredential, +): ResolvedAuth | undefined { + const key = credential?.key ?? context.env("GOOGLE_CLOUD_API_KEY"); + if (!key) return undefined; + const settings = vertexSettings(context, credential); + return { + auth: { apiKey: key }, + env: { + ...(settings.project === undefined ? {} : { GOOGLE_CLOUD_PROJECT: settings.project }), + ...(settings.location === undefined ? {} : { GOOGLE_CLOUD_LOCATION: settings.location }), + ...(settings.baseUrl === undefined ? {} : { GOOGLE_VERTEX_BASE_URL: settings.baseUrl }), + ...(settings.apiVersion === undefined + ? {} + : { GOOGLE_VERTEX_API_VERSION: settings.apiVersion }), + }, + source: credential?.key === undefined ? "GOOGLE_CLOUD_API_KEY" : "stored credential", + secretValues: [key], + }; +} + +export function createGoogleVertexStoredAuth(factories: CloudAuthFactories = {}): ApiKeyAuthMethod { + return { + displayName: "Google Vertex credentials", + login: async (interaction) => { + const method = await interaction.prompt({ + type: "select", + message: "Select Google Vertex AI authentication method:", + options: [ + { id: "api_key", label: "Google Cloud API key" }, + { id: "adc", label: "Application Default Credentials" }, + { id: "service_account", label: "Service account credentials file" }, + ], + }); + if (method === "api_key") { + return { + type: "api_key", + key: await interaction.prompt({ type: "secret", message: "Enter Google Cloud API key" }), + }; + } + if (method !== "adc" && method !== "service_account") { + throw new Error(`Unknown Google Vertex authentication method: ${method}`); + } + const project = await interaction.prompt({ + type: "text", + message: "Enter Google Cloud project ID:", + }); + const location = await interaction.prompt({ + type: "text", + message: "Enter Google Cloud location:", + }); + const keyFilename = + method === "service_account" + ? await interaction.prompt({ + type: "text", + message: "Enter service account credentials file path:", + }) + : undefined; + return { + type: "api_key", + env: { + GOOGLE_VERTEX_CREDENTIAL_TYPE: method, + GOOGLE_CLOUD_PROJECT: project, + GOOGLE_CLOUD_LOCATION: location, + ...(keyFilename === undefined ? {} : { GOOGLE_APPLICATION_CREDENTIALS: keyFilename }), + }, + }; + }, + resolve: async ({ context, credential, signal }) => { + const key = vertexApiKey(context, credential); + if (key !== undefined) return key; + if (credential === undefined) return undefined; + const mode = credential.env?.GOOGLE_VERTEX_CREDENTIAL_TYPE; + if (mode !== "adc" && mode !== "service_account") { + throw new AuthError( + "invalid_auth", + "google-vertex", + "Stored Google Vertex credential has no explicit authentication mode", + ); + } + if (mode === "service_account") { + const path = required( + credential.env?.GOOGLE_APPLICATION_CREDENTIALS, + "GOOGLE_APPLICATION_CREDENTIALS", + "google-vertex", + ); + if (!(await context.fileExists(path))) { + throw new AuthError( + "not_configured", + "google-vertex", + "Stored Google Vertex service account file is not readable", + ); + } + } + return googleAccessToken(context, credential, signal, factories, mode); + }, + }; +} + +export function createGoogleVertexSources( + factories: CloudAuthFactories = {}, +): readonly AmbientAuthSource[] { + const environment: AmbientAuthSource = { + type: "environment", + displayName: "Google Cloud API key", + resolve: async ({ context, signal }) => { + signal.throwIfAborted(); + return vertexApiKey(context); + }, + }; + const serviceAccount: AmbientAuthSource = { + type: "file", + displayName: "Google service account credentials", + resolve: async ({ context, signal }) => { + const path = context.env("GOOGLE_APPLICATION_CREDENTIALS")?.trim(); + if (!path) return undefined; + if (!(await context.fileExists(path))) { + throw new AuthError( + "not_configured", + "google-vertex", + "GOOGLE_APPLICATION_CREDENTIALS does not identify a readable file", + ); + } + return googleAccessToken(context, undefined, signal, factories, "service_account"); + }, + }; + const adc: AmbientAuthSource = { + type: "ambient", + displayName: "Google Application Default Credentials", + resolve: ({ context, signal }) => + googleAccessToken(context, undefined, signal, factories, "adc"), + }; + return [environment, serviceAccount, adc]; +} + +export function vertexRequestPolicy(resolved: ResolvedAuth): { + readonly credential: + | { readonly type: "api_key"; readonly apiKey: string } + | { readonly type: "adc"; readonly accessToken: string } + | { + readonly type: "service_account"; + readonly accessToken: string; + readonly credentialsFile: string; + }; + readonly project?: string; + readonly location?: string; + readonly baseUrl?: string; + readonly apiVersion?: string; +} { + const env: ProviderEnv = resolved.env ?? {}; + const common = { + ...(env.GOOGLE_CLOUD_PROJECT ? { project: env.GOOGLE_CLOUD_PROJECT } : {}), + ...(env.GOOGLE_CLOUD_LOCATION ? { location: env.GOOGLE_CLOUD_LOCATION } : {}), + ...(env.GOOGLE_VERTEX_BASE_URL ? { baseUrl: env.GOOGLE_VERTEX_BASE_URL } : {}), + ...(env.GOOGLE_VERTEX_API_VERSION ? { apiVersion: env.GOOGLE_VERTEX_API_VERSION } : {}), + }; + if (resolved.auth.apiKey) { + return { credential: { type: "api_key", apiKey: resolved.auth.apiKey }, ...common }; + } + const authorization = resolved.auth.headers?.authorization; + const token = authorization?.startsWith("Bearer ") ? authorization.slice(7) : undefined; + const accessToken = safeToken(token, "google-vertex"); + if (env.GOOGLE_VERTEX_CREDENTIAL_TYPE === "service_account") { + return { + credential: { + type: "service_account", + accessToken, + credentialsFile: required( + env.GOOGLE_APPLICATION_CREDENTIALS, + "GOOGLE_APPLICATION_CREDENTIALS", + "google-vertex", + ), + }, + ...common, + }; + } + if (env.GOOGLE_VERTEX_CREDENTIAL_TYPE !== "adc") { + throw new AuthError( + "invalid_auth", + "google-vertex", + "Google Vertex resolved an unknown credential type", + ); + } + return { credential: { type: "adc", accessToken }, ...common }; +} + +export { AZURE_SCOPE, GOOGLE_SCOPE }; diff --git a/packages/ai/src/credentials.ts b/packages/ai/src/credentials.ts index b7da7cd..17c581c 100644 --- a/packages/ai/src/credentials.ts +++ b/packages/ai/src/credentials.ts @@ -20,6 +20,8 @@ export interface OAuthCredential { readonly refresh: string; /** Access-token expiry as epoch milliseconds. */ readonly expiresAt: number; + /** Provider-scoped non-secret routing data kept only in the credential store. */ + readonly metadata?: ProviderEnv; } export type Credential = ApiKeyCredential | OAuthCredential; @@ -67,7 +69,7 @@ export function credentialSecretValues(credential: Credential): readonly string[ return [credential.access, credential.refresh]; } -function parseCredential(value: unknown, providerId: string): Credential { +export function validateCredential(value: unknown, providerId: string): Credential { const fail = (message: string): never => { throw new CredentialStoreError(`Stored credential for ${providerId} ${message}`); }; @@ -78,7 +80,7 @@ function parseCredential(value: unknown, providerId: string): Credential { credential.type === "api_key" ? new Set(["type", "key", "env"]) : credential.type === "oauth" - ? new Set(["type", "access", "refresh", "expiresAt"]) + ? new Set(["type", "access", "refresh", "expiresAt", "metadata"]) : undefined; if (allowed === undefined) return fail("has an unknown type"); for (const key of Object.keys(credential)) { @@ -118,6 +120,19 @@ function parseCredential(value: unknown, providerId: string): Credential { if (!Number.isSafeInteger(credential.expiresAt) || (credential.expiresAt as number) < 0) { fail("has an invalid expiresAt"); } + if (credential.metadata !== undefined) { + if ( + typeof credential.metadata !== "object" || + credential.metadata === null || + Array.isArray(credential.metadata) || + Object.getPrototypeOf(credential.metadata) !== Object.prototype || + Object.values(credential.metadata as Record).some( + (item) => typeof item !== "string", + ) + ) { + fail("has invalid oauth metadata"); + } + } return credential as unknown as OAuthCredential; } @@ -155,8 +170,12 @@ export class InMemoryCredentialStore implements CredentialStore { return this.enqueue(async () => { const current = this.credentials.get(providerId); const next = await fn(current); - if (next !== undefined) this.credentials.set(providerId, next); - return next ?? current; + if (next !== undefined) { + const validated = validateCredential(next, providerId); + this.credentials.set(providerId, validated); + return validated; + } + return current; }); } @@ -212,8 +231,9 @@ export class FileCredentialStore implements CredentialStore { const current = data[providerId]; const next = await fn(current); if (next === undefined) return current; - await this.persist({ ...data, [providerId]: next }); - return next; + const validated = validateCredential(next, providerId); + await this.persist({ ...data, [providerId]: validated }); + return validated; }), ); } @@ -259,7 +279,7 @@ export class FileCredentialStore implements CredentialStore { } const data: Record = {}; for (const [providerId, value] of Object.entries(parsed)) { - data[providerId] = parseCredential(value, providerId); + data[providerId] = validateCredential(value, providerId); } return data; } diff --git a/packages/ai/src/deepseek.ts b/packages/ai/src/deepseek.ts new file mode 100644 index 0000000..5f37c79 --- /dev/null +++ b/packages/ai/src/deepseek.ts @@ -0,0 +1,89 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createEnvironmentApiKeyAuth } from "./api-key-auth.ts"; +import { + type AuthContext, + AuthError, + createProviderAuthentication, + type ResolvedAuth, +} from "./auth.ts"; +import { getStaticModelCatalog } from "./catalog.ts"; +import type { CredentialStore } from "./credentials.ts"; +import type { ModelInfo } from "./model.ts"; +import { type OpenAiChatEndpoint, OpenAiChatProvider } from "./openai-chat-provider.ts"; +import { stripTrailingSlashes } from "./transport-safety.ts"; + +export const DEEPSEEK_PROVIDER_ID = "deepseek"; +export const DEEPSEEK_DISPLAY_NAME = "DeepSeek"; +export const DEEPSEEK_API_KEY_ENV = "DEEPSEEK_API_KEY"; +export const DEEPSEEK_BASE_URL = "https://api.deepseek.com"; + +export const DEEPSEEK_MODELS = getStaticModelCatalog(DEEPSEEK_PROVIDER_ID); + +export const deepSeekApiKeyAuth = createEnvironmentApiKeyAuth({ + providerId: DEEPSEEK_PROVIDER_ID, + displayName: "DeepSeek API key", + environmentVariables: [DEEPSEEK_API_KEY_ENV], +}); + +function endpointBaseUrl(model: ModelInfo): string { + if (model.endpoint?.type !== "fixed") { + throw new TypeError(`DeepSeek model ${model.modelId} requires a fixed endpoint`); + } + const url = new URL(model.endpoint.baseUrl); + if (url.protocol !== "https:" || url.username.length > 0 || url.password.length > 0) { + throw new TypeError(`DeepSeek model ${model.modelId} has an invalid endpoint`); + } + const baseUrl = stripTrailingSlashes(url.toString()); + if (baseUrl !== DEEPSEEK_BASE_URL) { + throw new TypeError(`DeepSeek model ${model.modelId} has an unexpected endpoint`); + } + return baseUrl; +} + +export const deepSeekEndpoint: OpenAiChatEndpoint = { + url: (model) => `${endpointBaseUrl(model)}/chat/completions`, + headers: (model, resolved) => { + endpointBaseUrl(model); + const key = resolved.auth.apiKey; + if (key === undefined || key.length === 0) { + throw new AuthError( + "invalid_auth", + DEEPSEEK_PROVIDER_ID, + "DeepSeek API key missing from resolved authentication", + ); + } + return { ...model.headers, Authorization: `Bearer ${key}` }; + }, +}; + +export interface DeepSeekProviderOptions { + readonly store: CredentialStore; + readonly context: AuthContext; + readonly models?: readonly ModelInfo[]; + readonly fetch?: typeof fetch; + readonly now?: () => number; +} + +/** Creates the built in DeepSeek provider without credential or network work. */ +export function createDeepSeekProvider(options: DeepSeekProviderOptions): OpenAiChatProvider { + const authentication = createProviderAuthentication({ + providerId: DEEPSEEK_PROVIDER_ID, + declaredMethods: ["environment", "file"], + methods: { apiKey: deepSeekApiKeyAuth }, + store: options.store, + context: options.context, + }); + return new OpenAiChatProvider({ + id: DEEPSEEK_PROVIDER_ID, + displayName: DEEPSEEK_DISPLAY_NAME, + authMethods: authentication.methods, + authentication, + endpoint: deepSeekEndpoint, + models: options.models ?? DEEPSEEK_MODELS, + resolveAuth: (signal: AbortSignal): Promise => authentication.resolve({ signal }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} diff --git a/packages/ai/src/diagnostics.ts b/packages/ai/src/diagnostics.ts new file mode 100644 index 0000000..319de77 --- /dev/null +++ b/packages/ai/src/diagnostics.ts @@ -0,0 +1,26 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { SafeProviderDiagnostic } from "@axl/protocol"; + +const REDACTED = "[REDACTED]"; +const MAX_DIAGNOSTIC_MESSAGE_LENGTH = 2_000; + +/** Redacts resolved credential values and bounds text before it reaches a diagnostic or event. */ +export function safeProviderMessage(message: string, secretValues: readonly string[] = []): string { + let safe = message; + const uniqueSecrets = [...new Set(secretValues.filter((value) => value.length > 0))].sort( + (left, right) => right.length - left.length, + ); + for (const secret of uniqueSecrets) safe = safe.split(secret).join(REDACTED); + return safe.slice(0, MAX_DIAGNOSTIC_MESSAGE_LENGTH); +} + +export function safeProviderDiagnostic( + code: string, + message: string, + severity: SafeProviderDiagnostic["severity"], + secretValues: readonly string[] = [], +): SafeProviderDiagnostic { + return { code, message: safeProviderMessage(message, secretValues), severity }; +} diff --git a/packages/ai/src/dialect.ts b/packages/ai/src/dialect.ts index 147fc02..807246e 100644 --- a/packages/ai/src/dialect.ts +++ b/packages/ai/src/dialect.ts @@ -116,12 +116,9 @@ export interface ProviderVisibleTool { readonly inputSchema: JsonObject; } -function renderName( - canonicalName: string, - override: string | undefined, - rule?: ToolNameRule, -): string { - const requested = override ?? canonicalName; +export function renderToolName(dialect: ToolDialectData, canonicalName: string): string { + const requested = dialect.tools?.[canonicalName]?.name ?? canonicalName; + const rule = dialect.nameRule; if (rule === undefined) return requested; const disallowed = new RegExp(`[^${rule.allowed}]+`, "g"); const sanitized = requested.replace(disallowed, "_").slice(0, rule.maxLength); @@ -157,7 +154,7 @@ export class FrozenToolRoster { const override = dialect.tools?.[tool.name]; const visible: ProviderVisibleTool = Object.freeze({ canonicalName: tool.name, - name: renderName(tool.name, override?.name, dialect.nameRule), + name: renderToolName(dialect, tool.name), description: override?.description ?? tool.description, inputSchema: override?.inputSchema ?? tool.inputSchema, }); diff --git a/packages/ai/src/fireworks.ts b/packages/ai/src/fireworks.ts new file mode 100644 index 0000000..893ebaf --- /dev/null +++ b/packages/ai/src/fireworks.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const FIREWORKS_PROVIDER_ID = "fireworks"; +export const FIREWORKS_API_KEY_ENV = "FIREWORKS_API_KEY"; +export const FIREWORKS_BASE_URL = "https://api.fireworks.ai/inference/v1"; + +export const FIREWORKS_PROVIDER_DEFINITION = { + id: FIREWORKS_PROVIDER_ID, + displayName: "Fireworks AI", + apiKeyDisplayName: "Fireworks API key", + environmentVariables: [FIREWORKS_API_KEY_ENV], + baseUrl: FIREWORKS_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createFireworksProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(FIREWORKS_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/gateway-messages.ts b/packages/ai/src/gateway-messages.ts new file mode 100644 index 0000000..86f09f9 --- /dev/null +++ b/packages/ai/src/gateway-messages.ts @@ -0,0 +1,668 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Transport-neutral Gateway messages request and SSE codec. + +import type { + JsonObject, + JsonValue, + ModelErrorCategory, + ModelStreamEvent, + Usage, +} from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { GatewayMessagesCompatibility, ModelInfo } from "./model.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + type PreparedRequestMessage, +} from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; + +export class GatewayMessagesCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "GatewayMessagesCodecError"; + } +} + +export interface EncodedGatewayMessagesRequest { + readonly body: JsonObject; +} + +export interface GatewayMessagesDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +type MutableJsonObject = Record; + +type GatewayToolState = { + readonly contentIndex: number; + readonly callId: string; + readonly name: string; + argumentsText: string; +}; + +const ERROR_CATEGORIES = new Set([ + "rate_limit", + "overloaded", + "network", + "timeout", + "authentication", + "authorization", + "invalid_request", + "context_limit", + "content_policy", + "provider_internal", + "stream_interrupted", + "unknown", +]); + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function compatibility(model: ModelInfo): GatewayMessagesCompatibility { + if (model.apiDialect !== "gateway-messages") { + throw new GatewayMessagesCodecError( + `Model ${model.modelId} does not use the gateway-messages dialect`, + ); + } + if (model.compatibility?.dialect !== "gateway-messages") { + throw new GatewayMessagesCodecError( + `Model ${model.modelId} has no Gateway messages compatibility record`, + ); + } + return model.compatibility; +} + +function requirePrepared(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new GatewayMessagesCodecError("Gateway messages requires a prepared model request"); + } +} + +function nonEmptyString(value: unknown, label: string): string { + if (typeof value !== "string" || value.length === 0) { + throw new GatewayMessagesCodecError(`${label} must be a non-empty string`); + } + return value; +} + +function nonNegativeInteger(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) < 0) { + throw new GatewayMessagesCodecError(`${label} must be a non-negative safe integer`); + } + return value as number; +} + +function nonNegativeNumber(value: unknown, label: string): number { + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new GatewayMessagesCodecError(`${label} must be a non-negative number`); + } + return value; +} + +function encodeContent( + request: PreparedModelRequest, + message: Extract, +): JsonValue[] { + return message.content.map((part) => { + if (part.type === "text") return { type: "text", text: part.text }; + const blob = request.preparation.blobs.get(part.blob.sha256); + if (blob === undefined) { + throw new GatewayMessagesCodecError(`Prepared blob ${part.blob.sha256} is unavailable`); + } + return { + type: "image", + data: Buffer.from(blob.bytes).toString("base64"), + mimeType: blob.reference.mediaType, + }; + }); +} + +function rejectTextContinuation( + continuation: Extract< + Extract["content"][number], + { type: "text" } + >["continuation"], + path: string, +): void { + if (continuation !== undefined) { + throw new GatewayMessagesCodecError(`${path} has unsupported continuation metadata`); + } +} + +function encodeAssistantMessage( + model: ModelInfo, + message: Extract, + messageIndex: number, +): JsonObject { + const content: JsonValue[] = message.content.map((part, contentIndex) => { + const path = `messages[${messageIndex}].content[${contentIndex}]`; + if (part.type === "blob") { + throw new GatewayMessagesCodecError(`${path} cannot replay an assistant image`); + } + if (part.type === "text") { + rejectTextContinuation(part.continuation, path); + return { + type: "text", + text: part.text, + ...(part.signature === undefined ? {} : { textSignature: part.signature.value }), + }; + } + return { + type: "thinking", + thinking: part.text, + ...(part.signature === undefined ? {} : { thinkingSignature: part.signature.value }), + ...(part.redacted === true ? { redacted: true } : {}), + }; + }); + const toolCalls = (message.toolCalls ?? []).map((call, callIndex): JsonValue => { + if (call.continuation?.responseId !== undefined || call.continuation?.itemId !== undefined) { + throw new GatewayMessagesCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported continuation metadata`, + ); + } + return { + type: "toolCall", + id: call.callId, + name: call.name, + arguments: call.input, + ...(call.signature === undefined ? {} : { thoughtSignature: call.signature.value }), + ...(call.continuation?.namespace === undefined + ? {} + : { namespace: call.continuation.namespace }), + }; + }); + if (message.continuation?.itemId !== undefined || message.continuation?.namespace !== undefined) { + throw new GatewayMessagesCodecError( + `messages[${messageIndex}] has unsupported continuation metadata`, + ); + } + if (content.length === 0 && toolCalls.length === 0) { + throw new GatewayMessagesCodecError( + `messages[${messageIndex}] has no Gateway-renderable assistant content`, + ); + } + const origin = message.origin; + return { + role: "assistant", + content: [...content, ...toolCalls], + api: "pi-messages", + provider: origin?.providerId ?? model.providerId, + model: model.modelId, + ...(origin !== undefined && origin.modelId !== model.modelId + ? { responseModel: origin.modelId } + : {}), + ...(message.continuation?.responseId === undefined + ? {} + : { responseId: message.continuation.responseId }), + usage: emptyGatewayUsage(), + stopReason: toolCalls.length > 0 ? "toolUse" : "stop", + timestamp: 0, + }; +} + +function encodeMessages(model: ModelInfo, request: PreparedModelRequest): JsonValue[] { + return request.messages.map((message, messageIndex): JsonValue => { + if (message.role === "user") { + const content = encodeContent(request, message); + return { + role: "user", + content: + content.length === 1 && object(content[0])?.type === "text" + ? (object(content[0])?.text as string) + : content, + timestamp: 0, + }; + } + if (message.role === "assistant") { + return encodeAssistantMessage(model, message, messageIndex); + } + return { + role: "toolResult", + toolCallId: message.callId, + toolName: message.name, + content: encodeContent(request, message), + isError: message.isError, + timestamp: 0, + }; + }); +} + +function encodeTools(request: PreparedModelRequest): JsonValue[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + return request.tools.map((tool): JsonValue => { + if (tool.preparedConstraint?.type === "grammar") { + throw new GatewayMessagesCodecError( + `Gateway messages cannot render grammar-constrained tool ${tool.canonicalName}`, + ); + } + return { + name: tool.name, + description: tool.description, + parameters: tool.inputSchema, + ...(tool.preparedConstraint?.type === "json-schema" + ? { + constrainedSampling: { + type: "json_schema", + strict: tool.preparedConstraint.strict ? "require" : "prefer", + }, + } + : {}), + }; + }); +} + +function emptyGatewayUsage(): JsonObject { + return { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 0, + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, + }; +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeGatewayMessagesRequest( + model: ModelInfo, + request: PreparedModelRequest, +): EncodedGatewayMessagesRequest { + requirePrepared(request); + compatibility(model); + if (request.modelId !== model.modelId) { + throw new GatewayMessagesCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (request.safetySettings !== undefined) { + throw new GatewayMessagesCodecError("Gateway messages cannot render safety settings"); + } + if (request.sampling !== undefined) { + throw new GatewayMessagesCodecError("Gateway messages cannot render sampling controls"); + } + const context: MutableJsonObject = { messages: encodeMessages(model, request) }; + if (request.system !== undefined) context.systemPrompt = request.system; + const tools = request.toolChoice === "none" ? undefined : encodeTools(request); + if (tools !== undefined) context.tools = tools; + if (request.toolChoice !== undefined && request.toolChoice !== "none" && tools === undefined) { + throw new GatewayMessagesCodecError(`toolChoice ${request.toolChoice} needs at least one tool`); + } + const options: MutableJsonObject = {}; + if ( + request.preparation.reasoning !== undefined && + request.preparation.reasoning.effective !== "off" + ) { + options.reasoning = request.preparation.reasoning.effective; + } + if (request.maxOutputTokens !== undefined) options.maxTokens = request.maxOutputTokens; + if (request.toolChoice !== undefined) options.toolChoice = request.toolChoice; + if (request.preparation.cache.retention !== "none") { + options.cacheRetention = request.preparation.cache.retention; + if (request.preparation.cache.sessionId !== undefined) { + options.sessionId = request.preparation.cache.sessionId; + } + } + if (request.metadata !== undefined) options.metadata = { ...request.metadata }; + return { body: { model: model.modelId, context, options } }; +} + +function usage(raw: unknown): Usage { + const value = object(raw); + if (value === undefined) throw new GatewayMessagesCodecError("Gateway usage must be an object"); + const input = nonNegativeInteger(value.input, "Gateway usage input"); + const output = nonNegativeInteger(value.output, "Gateway usage output"); + const cacheRead = nonNegativeInteger(value.cacheRead, "Gateway usage cacheRead"); + const cacheWrite = nonNegativeInteger(value.cacheWrite, "Gateway usage cacheWrite"); + if (value.reasoning !== undefined) { + nonNegativeInteger(value.reasoning, "Gateway usage reasoning"); + } + nonNegativeInteger(value.totalTokens, "Gateway usage totalTokens"); + const cost = object(value.cost); + if (cost === undefined) + throw new GatewayMessagesCodecError("Gateway usage cost must be an object"); + for (const field of ["input", "output", "cacheRead", "cacheWrite", "total"] as const) { + nonNegativeNumber(cost[field], `Gateway usage cost ${field}`); + } + return { + inputTokens: input, + outputTokens: output, + cacheReadTokens: cacheRead, + cacheWriteTokens: cacheWrite, + ...(value.reasoning === undefined ? {} : { reasoningTokens: value.reasoning as number }), + costUsd: cost.total as number, + }; +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function responseMetadata( + event: Record, + options: GatewayMessagesDecodeOptions, +): NonNullable["response"]> { + if (event.requestedModelId !== undefined && event.requestedModelId !== options.request.modelId) { + throw new GatewayMessagesCodecError("Gateway terminal event has a mismatched requested model"); + } + const routedModel = event.routedModelId ?? event.responseModel; + const responseId = event.responseId; + const nativeStopReason = event.nativeStopReason ?? event.rawStopReason ?? event.reason; + if (routedModel !== undefined) nonEmptyString(routedModel, "Gateway routed model"); + if (responseId !== undefined) nonEmptyString(responseId, "Gateway response ID"); + if (nativeStopReason !== undefined) + nonEmptyString(nativeStopReason, "Gateway native stop reason"); + return { + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModel === undefined ? {} : { routedModelId: routedModel as string }), + ...(responseId === undefined ? {} : { responseId: responseId as string }), + ...(nativeStopReason === undefined ? {} : { nativeStopReason: nativeStopReason as string }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }; +} + +function replayMetadata( + target: "text" | "thinking", + contentIndex: number, + signature: string, + options: GatewayMessagesDecodeOptions, + redacted?: boolean, +): ModelStreamEvent { + return { + type: "replay_metadata", + target, + contentIndex, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.model.modelId, + signature, + ...(redacted === true ? { redacted: true } : {}), + }; +} + +function errorCategory(value: unknown): ModelErrorCategory | undefined { + if (value === undefined) return undefined; + if (typeof value !== "string" || !ERROR_CATEGORIES.has(value as ModelErrorCategory)) { + throw new GatewayMessagesCodecError("Gateway error category is malformed"); + } + return value as ModelErrorCategory; +} + +/** Decodes framed Gateway SSE values into canonical model stream events. */ +export async function* decodeGatewayMessagesStream( + frames: AsyncIterable, + options: GatewayMessagesDecodeOptions, +): AsyncGenerator { + requirePrepared(options.request); + compatibility(options.model); + const tools = new Map(); + const content = new Map(); + let emittedContent = false; + + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (frame.data === "[DONE]") break; + let event: Record; + try { + const parsed = object(JSON.parse(frame.data) as unknown); + if (parsed === undefined) throw new Error("frame is not an object"); + event = parsed; + } catch (cause) { + throw new GatewayMessagesCodecError("Gateway sent an undecodable stream frame", { cause }); + } + const type = nonEmptyString(event.type, "Gateway event type"); + if (type === "start") continue; + if (type === "text_start" || type === "thinking_start") { + const contentIndex = nonNegativeInteger(event.contentIndex, `Gateway ${type} contentIndex`); + if (content.has(contentIndex)) { + throw new GatewayMessagesCodecError(`Gateway repeated content position ${contentIndex}`); + } + content.set(contentIndex, { + type: type === "text_start" ? "text" : "thinking", + text: "", + }); + continue; + } + if (type === "text_delta" || type === "thinking_delta") { + const contentIndex = nonNegativeInteger(event.contentIndex, `Gateway ${type} contentIndex`); + const contentType = type === "text_delta" ? "text" : "thinking"; + const delta = typeof event.delta === "string" ? event.delta : event.text; + if (typeof delta !== "string") { + throw new GatewayMessagesCodecError(`Gateway ${type} delta must be a string`); + } + const state = content.get(contentIndex) ?? { type: contentType, text: "" }; + if (state.type !== contentType) { + throw new GatewayMessagesCodecError(`Gateway changed content type at ${contentIndex}`); + } + state.text += delta; + content.set(contentIndex, state); + if (delta.length > 0) emittedContent = true; + yield { + type: contentType === "text" ? "text_delta" : "thinking_delta", + text: delta, + contentIndex, + }; + continue; + } + if (type === "text_end" || type === "thinking_end") { + const contentIndex = nonNegativeInteger(event.contentIndex, `Gateway ${type} contentIndex`); + const contentType = type === "text_end" ? "text" : "thinking"; + const finalContent = event.content; + if (typeof finalContent !== "string") { + throw new GatewayMessagesCodecError(`Gateway ${type} content must be a string`); + } + const state = content.get(contentIndex) ?? { type: contentType, text: "" }; + if (state.type !== contentType || !finalContent.startsWith(state.text)) { + throw new GatewayMessagesCodecError(`Gateway ${type} does not match streamed content`); + } + const remainder = finalContent.slice(state.text.length); + if (remainder.length > 0) { + emittedContent = true; + yield { + type: contentType === "text" ? "text_delta" : "thinking_delta", + text: remainder, + contentIndex, + }; + } + content.delete(contentIndex); + const signature = event.contentSignature; + if (event.redacted !== undefined && typeof event.redacted !== "boolean") { + throw new GatewayMessagesCodecError(`Gateway ${type} redacted must be a boolean`); + } + if (type === "text_end" && event.redacted !== undefined) { + throw new GatewayMessagesCodecError("Gateway text_end cannot be redacted"); + } + if (event.redacted === true && signature === undefined) { + throw new GatewayMessagesCodecError("Gateway redacted thinking requires a signature"); + } + if (signature !== undefined) { + nonEmptyString(signature, `Gateway ${type} contentSignature`); + emittedContent = true; + yield replayMetadata( + contentType, + contentIndex, + signature as string, + options, + contentType === "thinking" && event.redacted === true, + ); + } + continue; + } + if (type === "toolcall_start") { + const contentIndex = nonNegativeInteger(event.contentIndex, "Gateway tool contentIndex"); + const callId = nonEmptyString(event.id ?? event.callId, "Gateway tool call ID"); + const name = nonEmptyString(event.toolName ?? event.name, "Gateway tool name"); + if (tools.has(contentIndex)) { + throw new GatewayMessagesCodecError(`Gateway repeated tool call position ${contentIndex}`); + } + tools.set(contentIndex, { contentIndex, callId, name, argumentsText: "" }); + emittedContent = true; + yield { + type: "tool_call_start", + contentIndex, + callId, + name: reverseToolName(options.request, name), + }; + continue; + } + if (type === "toolcall_delta") { + const contentIndex = nonNegativeInteger(event.contentIndex, "Gateway tool contentIndex"); + if (typeof event.delta !== "string") { + throw new GatewayMessagesCodecError("Gateway tool call delta must be a string"); + } + const state = tools.get(contentIndex); + if (state === undefined) { + throw new GatewayMessagesCodecError("Gateway tool call delta has no matching start"); + } + state.argumentsText += event.delta; + yield { + type: "tool_call_delta", + contentIndex, + callId: state.callId, + argumentsDelta: event.delta, + }; + continue; + } + if (type === "toolcall_end") { + const contentIndex = nonNegativeInteger(event.contentIndex, "Gateway tool contentIndex"); + const state = tools.get(contentIndex); + if (state === undefined) { + throw new GatewayMessagesCodecError("Gateway tool call end has no matching start"); + } + const toolCall = object(event.toolCall); + if (toolCall === undefined) { + throw new GatewayMessagesCodecError("Gateway tool call end is malformed"); + } + const callId = nonEmptyString(toolCall.id, "Gateway completed tool call ID"); + const name = nonEmptyString(toolCall.name, "Gateway completed tool name"); + if (callId !== state.callId || name !== state.name) { + throw new GatewayMessagesCodecError("Gateway completed tool call does not match its start"); + } + const input = object(toolCall.arguments); + if (input === undefined) { + throw new GatewayMessagesCodecError("Gateway completed tool arguments must be an object"); + } + if (state.argumentsText.length > 0) { + try { + if (object(JSON.parse(state.argumentsText) as unknown) === undefined) { + throw new Error("arguments are not an object"); + } + } catch (cause) { + throw new GatewayMessagesCodecError("Gateway streamed tool arguments are malformed", { + cause, + }); + } + } + tools.delete(contentIndex); + const canonicalName = reverseToolName(options.request, name); + yield { + type: "tool_call", + contentIndex, + callId, + name: canonicalName, + input: input as JsonObject, + }; + const signature = toolCall.thoughtSignature; + const namespace = toolCall.namespace; + if (signature !== undefined) { + nonEmptyString(signature, "Gateway tool signature"); + } + if (namespace !== undefined) { + nonEmptyString(namespace, "Gateway tool namespace"); + } + if (signature !== undefined || namespace !== undefined) { + yield { + type: "replay_metadata", + target: "tool_call", + contentIndex, + callId, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.model.modelId, + ...(signature === undefined ? {} : { signature: signature as string }), + ...(namespace === undefined ? {} : { namespace: namespace as string }), + }; + } + continue; + } + if (type === "done") { + if (tools.size > 0) + throw new GatewayMessagesCodecError("Gateway ended with incomplete tool calls"); + const reason = nonEmptyString(event.reason, "Gateway stop reason"); + const stopReason = + reason === "stop" + ? "stop" + : reason === "length" + ? "length" + : reason === "toolUse" || reason === "tool_use" + ? "tool_use" + : undefined; + if (stopReason === undefined) { + throw new GatewayMessagesCodecError(`Gateway stop reason ${reason} is unsupported`); + } + yield { + type: "completed", + stopReason, + usage: usage(event.usage), + ...(stopReason === "length" ? { partial: true } : {}), + response: responseMetadata(event, options), + }; + return; + } + if (type === "error") { + const reason = nonEmptyString(event.reason, "Gateway error reason"); + if (reason === "aborted") { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (reason !== "error") { + throw new GatewayMessagesCodecError(`Gateway error reason ${reason} is unsupported`); + } + const code = + event.code === undefined + ? "gateway_error" + : nonEmptyString(event.code, "Gateway error code"); + if (event.retryable !== undefined && typeof event.retryable !== "boolean") { + throw new GatewayMessagesCodecError("Gateway error retryable must be a boolean"); + } + const category = errorCategory(event.category) ?? "provider_internal"; + yield { + type: "error", + code, + message: safeProviderMessage( + typeof event.errorMessage === "string" + ? event.errorMessage + : "Gateway reported a failure", + options.secretValues, + ), + retryable: event.retryable === true, + category, + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(event, options), + }; + return; + } + throw new GatewayMessagesCodecError(`Gateway event type ${type} is unsupported`); + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + } +} diff --git a/packages/ai/src/google-generative-ai.ts b/packages/ai/src/google-generative-ai.ts new file mode 100644 index 0000000..c9f32ca --- /dev/null +++ b/packages/ai/src/google-generative-ai.ts @@ -0,0 +1,35 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Google Generative AI dialect boundary over the shared Google codec. + +import type { ModelInfo, ModelStreamEvent } from "./model.ts"; +import { + decodeGoogleStream, + type EncodedGoogleRequest, + encodeGoogleRequest, + GoogleGenerativeAiCodecError, + type GoogleDecodeOptions, +} from "./google-shared.ts"; +import type { PreparedModelRequest } from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; + +export { GoogleGenerativeAiCodecError }; +export type EncodedGoogleGenerativeAiRequest = EncodedGoogleRequest; +export type GoogleGenerativeAiDecodeOptions = GoogleDecodeOptions; + +/** Encodes only the Google Generative AI dialect. */ +export function encodeGoogleGenerativeAiRequest( + model: ModelInfo, + request: PreparedModelRequest, +): EncodedGoogleGenerativeAiRequest { + return encodeGoogleRequest(model, request, "google-generative-ai"); +} + +/** Decodes only the Google Generative AI dialect. */ +export function decodeGoogleGenerativeAiStream( + frames: AsyncIterable, + options: GoogleGenerativeAiDecodeOptions, +): AsyncGenerator { + return decodeGoogleStream(frames, options, "google-generative-ai"); +} diff --git a/packages/ai/src/google-shared.ts b/packages/ai/src/google-shared.ts new file mode 100644 index 0000000..06090d5 --- /dev/null +++ b/packages/ai/src/google-shared.ts @@ -0,0 +1,804 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Shared Axl-native Google request and streaming response codec. + +import type { JsonObject, JsonValue, Usage } from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { + GoogleGenerativeAiCompatibility, + GoogleVertexCompatibility, + ModelInfo, + ModelStreamEvent, +} from "./model.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + type PreparedRequestMessage, +} from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; +import { withUsageCost } from "./usage.ts"; + +const SUPPORTED_IMAGE_TYPES = new Set(["image/jpeg", "image/png", "image/gif", "image/webp"]); +const RESERVED_GENERATION_FIELDS = new Set([ + "temperature", + "topP", + "topK", + "candidateCount", + "maxOutputTokens", + "responseMimeType", + "responseSchema", + "seed", + "thinkingConfig", +]); +const SAFETY_FINISH_REASONS = new Set([ + "BLOCKLIST", + "PROHIBITED_CONTENT", + "SPII", + "SAFETY", + "IMAGE_SAFETY", + "IMAGE_PROHIBITED_CONTENT", + "IMAGE_RECITATION", + "IMAGE_OTHER", + "RECITATION", + "LANGUAGE", +]); + +export class GoogleGenerativeAiCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "GoogleGenerativeAiCodecError"; + } +} + +export interface EncodedGoogleRequest { + readonly modelId: string; + readonly body: JsonObject; + /** Protocol headers only. Authentication remains transport-owned. */ + readonly headers: Readonly>; +} + +type MutableJsonObject = Record; + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +export type GoogleApiDialect = "google-generative-ai" | "google-vertex"; +type GoogleCompatibility = GoogleGenerativeAiCompatibility | GoogleVertexCompatibility; + +function compatibility(model: ModelInfo, expectedDialect: GoogleApiDialect): GoogleCompatibility { + if (model.apiDialect !== expectedDialect) { + throw new GoogleGenerativeAiCodecError( + `Model ${model.modelId} does not use the ${expectedDialect} dialect`, + ); + } + if (model.compatibility?.dialect !== expectedDialect) { + throw new GoogleGenerativeAiCodecError( + `Model ${model.modelId} has no ${expectedDialect} compatibility record`, + ); + } + return model.compatibility; +} + +function preparedRequest(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new GoogleGenerativeAiCodecError( + "Google Generative AI requires a prepared model request", + ); + } +} + +function isGemini3(modelId: string): boolean { + return /^gemini(?:-live)?-3(?:\.|-)/i.test(modelId); +} + +function isGemini3Pro(modelId: string): boolean { + return /^gemini(?:-live)?-3(?:\.\d+)?-pro/i.test(modelId); +} + +function isGemini3Flash(modelId: string): boolean { + const id = modelId.toLowerCase(); + return ( + /^gemini(?:-live)?-3(?:\.\d+)?-flash/.test(id) || + id === "gemini-flash-latest" || + id === "gemini-flash-lite-latest" + ); +} + +function isGemma4(modelId: string): boolean { + return /gemma-?4/i.test(modelId); +} + +function requiresToolCallId(modelId: string): boolean { + return isGemini3(modelId) || modelId.startsWith("claude-") || modelId.startsWith("gpt-oss-"); +} + +function googleSignature(value: string, path: string): string { + if (value.length % 4 !== 0 || !/^[A-Za-z0-9+/]+={0,2}$/.test(value)) { + throw new GoogleGenerativeAiCodecError(`${path} is not a valid Google thought signature`); + } + return value; +} + +function verifiedImage(request: PreparedModelRequest, sha256: string): MutableJsonObject { + const blob = request.preparation.blobs.get(sha256); + if (blob === undefined) { + throw new GoogleGenerativeAiCodecError(`Prepared blob ${sha256} is unavailable`); + } + if (!SUPPORTED_IMAGE_TYPES.has(blob.reference.mediaType)) { + throw new GoogleGenerativeAiCodecError( + `Google Generative AI does not support image media type ${blob.reference.mediaType}`, + ); + } + return { + inlineData: { + mimeType: blob.reference.mediaType, + data: Buffer.from(blob.bytes).toString("base64"), + }, + }; +} + +function basicParts( + request: PreparedModelRequest, + message: PreparedRequestMessage, +): MutableJsonObject[] { + return message.content.map((content) => { + if (content.type === "text") return { text: content.text }; + if (content.type === "blob") return verifiedImage(request, content.blob.sha256); + throw new GoogleGenerativeAiCodecError("Google Generative AI cannot encode this content block"); + }); +} + +function rejectContinuation(message: PreparedRequestMessage, messageIndex: number): void { + if (message.role !== "assistant") return; + if (message.continuation !== undefined) { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}] has continuation metadata unsupported by Google Generative AI`, + ); + } + for (const [contentIndex, content] of message.content.entries()) { + if (content.type === "text" && content.continuation !== undefined) { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported continuation metadata`, + ); + } + } + for (const [callIndex, call] of (message.toolCalls ?? []).entries()) { + if (call.continuation !== undefined) { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported continuation metadata`, + ); + } + } +} + +function assistantParts( + model: ModelInfo, + message: Extract, + messageIndex: number, +): MutableJsonObject[] { + rejectContinuation(message, messageIndex); + const parts: MutableJsonObject[] = []; + for (const content of message.content) { + if (content.type === "blob") { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}] cannot replay an assistant image through Google Generative AI`, + ); + } + if (content.type === "text") { + if (content.text.length === 0 && content.signature === undefined) continue; + parts.push({ + text: content.text, + ...(content.signature === undefined + ? {} + : { + thoughtSignature: googleSignature( + content.signature.value, + `messages[${messageIndex}] text signature`, + ), + }), + }); + continue; + } + if (content.redacted === true) { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}] contains unsupported redacted thinking`, + ); + } + if (content.text.length === 0 && content.signature === undefined) continue; + if (content.signature === undefined) { + if (content.text.length > 0) parts.push({ text: content.text }); + continue; + } + parts.push({ + thought: true, + text: content.text, + thoughtSignature: googleSignature( + content.signature.value, + `messages[${messageIndex}] thinking signature`, + ), + }); + } + for (const call of message.toolCalls ?? []) { + parts.push({ + functionCall: { + name: call.name, + args: call.input, + ...(requiresToolCallId(model.modelId) ? { id: call.callId } : {}), + }, + ...(call.signature === undefined + ? {} + : { + thoughtSignature: googleSignature( + call.signature.value, + `messages[${messageIndex}] tool signature`, + ), + }), + }); + } + if (parts.length === 0) { + throw new GoogleGenerativeAiCodecError( + `messages[${messageIndex}] has no Google-renderable assistant content`, + ); + } + return parts; +} + +function toolResultParts( + model: ModelInfo, + request: PreparedModelRequest, + message: Extract, +): { readonly response: MutableJsonObject; readonly imageTurn?: MutableJsonObject } { + const texts = message.content.filter((content) => content.type === "text"); + const images = message.content.filter((content) => content.type === "blob"); + const text = texts.map((content) => content.text).join("\n"); + const imageParts = images.map((content) => verifiedImage(request, content.blob.sha256)); + const responseText = text.length > 0 ? text : imageParts.length > 0 ? "(see attached image)" : ""; + const nestedImages = imageParts.length > 0 && isGemini3(model.modelId); + return { + response: { + functionResponse: { + name: message.name, + response: message.isError ? { error: responseText } : { output: responseText }, + ...(requiresToolCallId(model.modelId) ? { id: message.callId } : {}), + ...(nestedImages ? { parts: imageParts } : {}), + }, + }, + ...(imageParts.length > 0 && !nestedImages + ? { imageTurn: { role: "user", parts: [{ text: "Tool result image:" }, ...imageParts] } } + : {}), + }; +} + +function encodeContents(model: ModelInfo, request: PreparedModelRequest): JsonValue[] { + const contents: MutableJsonObject[] = []; + for (const [messageIndex, message] of request.messages.entries()) { + if (message.role === "user") { + contents.push({ role: "user", parts: basicParts(request, message) }); + continue; + } + if (message.role === "assistant") { + contents.push({ role: "model", parts: assistantParts(model, message, messageIndex) }); + continue; + } + const result = toolResultParts(model, request, message); + const previous = contents.at(-1); + if ( + previous?.role === "user" && + Array.isArray(previous.parts) && + previous.parts.some((part) => object(part)?.functionResponse !== undefined) + ) { + previous.parts.push(result.response); + } else { + contents.push({ role: "user", parts: [result.response] }); + } + if (result.imageTurn !== undefined) contents.push(result.imageTurn); + } + return contents; +} + +function encodeTools( + model: ModelInfo, + request: PreparedModelRequest, + expectedDialect: GoogleApiDialect, +): JsonValue[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + const strictSupported = compatibility(model, expectedDialect).supportsStrictTools === true; + const declarations = request.tools.map((tool): JsonValue => { + if (tool.preparedConstraint?.type === "grammar") { + throw new GoogleGenerativeAiCodecError( + `Google Generative AI cannot render grammar-constrained tool ${tool.canonicalName}`, + ); + } + if (tool.preparedConstraint?.type === "json-schema" && tool.preparedConstraint.strict) { + if (!strictSupported) { + throw new GoogleGenerativeAiCodecError( + `Google Generative AI strict tool ${tool.canonicalName} is unsupported by this model`, + ); + } + } + return { + name: tool.name, + description: tool.description, + parametersJsonSchema: tool.inputSchema, + }; + }); + return [{ functionDeclarations: declarations }]; +} + +function applyThinking( + body: MutableJsonObject, + model: ModelInfo, + request: PreparedModelRequest, +): void { + const reasoning = request.preparation.reasoning; + if (reasoning === undefined) return; + if (reasoning.effective === "off") { + if (isGemini3Pro(model.modelId)) { + body.generationConfig = { + ...(object(body.generationConfig) as JsonObject), + thinkingConfig: { thinkingLevel: "LOW" }, + }; + } else if (isGemini3Flash(model.modelId) || isGemma4(model.modelId)) { + body.generationConfig = { + ...(object(body.generationConfig) as JsonObject), + thinkingConfig: { thinkingLevel: "MINIMAL" }, + }; + } else { + body.generationConfig = { + ...(object(body.generationConfig) as JsonObject), + thinkingConfig: { thinkingBudget: 0 }, + }; + } + return; + } + const config: MutableJsonObject = { includeThoughts: true }; + if (reasoning.tokenBudget !== undefined) { + config.thinkingBudget = reasoning.tokenBudget; + } else { + const value = (reasoning.providerValue ?? reasoning.effective).toUpperCase(); + if (!new Set(["MINIMAL", "LOW", "MEDIUM", "HIGH"]).has(value)) { + throw new GoogleGenerativeAiCodecError(`Unsupported Google thinking level ${value}`); + } + config.thinkingLevel = value; + } + body.generationConfig = { + ...(object(body.generationConfig) as JsonObject), + thinkingConfig: config, + }; +} + +function validCachedContentName(name: string, dialect: GoogleApiDialect): boolean { + if (/^cachedContents\/[A-Za-z0-9._~-]+$/.test(name)) return true; + return ( + dialect === "google-vertex" && + /^projects\/[A-Za-z0-9._~-]+\/locations\/[A-Za-z0-9._~-]+\/cachedContents\/[A-Za-z0-9._~-]+$/.test( + name, + ) + ); +} + +function applySampling(body: MutableJsonObject, request: PreparedModelRequest): void { + const generation = (object(body.generationConfig) ?? {}) as MutableJsonObject; + if (request.maxOutputTokens !== undefined) generation.maxOutputTokens = request.maxOutputTokens; + const sampling = request.sampling; + if (sampling !== undefined) { + if (sampling.temperature !== undefined) generation.temperature = sampling.temperature; + if (sampling.topP !== undefined) generation.topP = sampling.topP; + if (sampling.topK !== undefined) generation.topK = sampling.topK; + if (sampling.seed !== undefined) generation.seed = sampling.seed; + for (const [field, value] of Object.entries(sampling.custom ?? {})) { + if (RESERVED_GENERATION_FIELDS.has(field) || field in generation) { + throw new GoogleGenerativeAiCodecError( + `Custom sampling field ${field} collides with a Google generation field`, + ); + } + generation[field] = value; + } + } + if (Object.keys(generation).length > 0) body.generationConfig = generation; +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeGoogleRequest( + model: ModelInfo, + request: PreparedModelRequest, + expectedDialect: GoogleApiDialect, +): EncodedGoogleRequest { + preparedRequest(request); + compatibility(model, expectedDialect); + if (request.modelId !== model.modelId) { + throw new GoogleGenerativeAiCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (request.metadata !== undefined && Object.keys(request.metadata).length > 0) { + throw new GoogleGenerativeAiCodecError("Google Generative AI request metadata is unsupported"); + } + if (request.preparation.cache.retention === "long") { + throw new GoogleGenerativeAiCodecError( + "Google Generative AI long cache retention is unsupported", + ); + } + + const body: MutableJsonObject = { contents: encodeContents(model, request) }; + if (request.system !== undefined && request.system.length > 0) { + body.systemInstruction = { parts: [{ text: request.system }] }; + } + const tools = encodeTools(model, request, expectedDialect); + if (tools !== undefined) body.tools = tools; + if (request.toolChoice !== undefined) { + if (request.toolChoice !== "none" && tools === undefined) { + throw new GoogleGenerativeAiCodecError( + `toolChoice ${request.toolChoice} needs at least one tool`, + ); + } + const strict = request.tools?.some( + (tool) => tool.preparedConstraint?.type === "json-schema" && tool.preparedConstraint.strict, + ); + body.toolConfig = { + functionCallingConfig: { + mode: + request.toolChoice === "required" + ? "ANY" + : request.toolChoice === "none" + ? "NONE" + : strict + ? "VALIDATED" + : "AUTO", + }, + }; + } else if ( + request.tools?.some( + (tool) => tool.preparedConstraint?.type === "json-schema" && tool.preparedConstraint.strict, + ) + ) { + body.toolConfig = { functionCallingConfig: { mode: "VALIDATED" } }; + } + if (request.safetySettings !== undefined) { + body.safetySettings = request.safetySettings.map((setting) => ({ ...setting })); + } + if (request.preparation.cache.sessionId !== undefined) { + if (!validCachedContentName(request.preparation.cache.sessionId, expectedDialect)) { + throw new GoogleGenerativeAiCodecError( + "Google cached content must use a cachedContents resource name", + ); + } + body.cachedContent = request.preparation.cache.sessionId; + } + applySampling(body, request); + applyThinking(body, model, request); + return { + modelId: model.modelId, + body, + headers: { accept: "text/event-stream", "content-type": "application/json" }, + }; +} + +function parseFrame(frame: SseFrame): Record | undefined { + if (frame.data.trim() === "[DONE]") return undefined; + if (frame.event !== undefined && frame.event !== "message" && frame.event !== "error") { + return undefined; + } + try { + const value = object(JSON.parse(frame.data) as unknown); + if (value === undefined) throw new Error("frame is not an object"); + return value; + } catch (error) { + throw new GoogleGenerativeAiCodecError( + "Provider sent an undecodable Google Generative AI stream frame", + { cause: error }, + ); + } +} + +function nonNegative(value: unknown, name: string): number { + if (value === undefined || value === null) return 0; + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new GoogleGenerativeAiCodecError(`Google usage ${name} must be non-negative`); + } + return value; +} + +function usage(raw: unknown, model: ModelInfo): Usage { + const value = object(raw); + if (value === undefined) throw new GoogleGenerativeAiCodecError("Google usage must be an object"); + const prompt = nonNegative(value.promptTokenCount, "promptTokenCount"); + const cached = nonNegative(value.cachedContentTokenCount, "cachedContentTokenCount"); + if (cached > prompt) { + throw new GoogleGenerativeAiCodecError("Google cached input tokens exceed prompt tokens"); + } + const reasoning = nonNegative(value.thoughtsTokenCount, "thoughtsTokenCount"); + const mapped: Usage = { + inputTokens: prompt - cached, + outputTokens: nonNegative(value.candidatesTokenCount, "candidatesTokenCount") + reasoning, + cacheReadTokens: cached, + cacheWriteTokens: 0, + reasoningTokens: reasoning, + }; + return model.cost === undefined ? mapped : withUsageCost(model.cost, mapped); +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function retryableProviderCode(code: string): boolean { + return new Set(["429", "RESOURCE_EXHAUSTED", "UNAVAILABLE", "INTERNAL", "DEADLINE_EXCEEDED"]).has( + code, + ); +} + +interface ActiveTextBlock { + readonly type: "text" | "thinking"; + readonly contentIndex: number; + signature?: string; +} + +export interface GoogleDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +/** Decodes Google Generative AI SSE frames into canonical stream events. */ +export async function* decodeGoogleStream( + frames: AsyncIterable, + options: GoogleDecodeOptions, + expectedDialect: GoogleApiDialect, +): AsyncGenerator { + preparedRequest(options.request); + compatibility(options.model, expectedDialect); + let responseId: string | undefined; + let routedModelId: string | undefined; + let nativeStopReason: string | undefined; + let finalUsage: Usage = { + inputTokens: 0, + outputTokens: 0, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + ...(options.model.cost === undefined ? {} : { costUsd: 0 }), + }; + let active: ActiveTextBlock | undefined; + let nextContentIndex = 0; + let generatedCallId = 0; + const callIds = new Set(); + let emittedContent = false; + let emittedToolCall = false; + + const responseMetadata = () => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined ? {} : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + ...(nativeStopReason === undefined ? {} : { nativeStopReason }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }); + const replayForActive = (): ModelStreamEvent | undefined => { + if (active?.signature === undefined) return undefined; + return { + type: "replay_metadata", + target: active.type, + contentIndex: active.contentIndex, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.request.modelId, + signature: active.signature, + }; + }; + + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + const chunk = parseFrame(frame); + if (chunk === undefined) continue; + + if (chunk.error !== undefined || frame.event === "error") { + const providerError = object(chunk.error) ?? chunk; + const code = String(providerError.status ?? providerError.code ?? "provider_error"); + yield { + type: "error", + code, + message: safeProviderMessage( + typeof providerError.message === "string" + ? providerError.message + : "Google Generative AI reported a failure", + options.secretValues, + ), + retryable: retryableProviderCode(code), + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + + if (typeof chunk.responseId === "string" && chunk.responseId.length > 0) { + responseId ??= chunk.responseId; + } + if (typeof chunk.modelVersion === "string" && chunk.modelVersion.length > 0) { + routedModelId = chunk.modelVersion; + } + if (chunk.usageMetadata !== undefined) finalUsage = usage(chunk.usageMetadata, options.model); + + const promptFeedback = object(chunk.promptFeedback); + const blockReason = promptFeedback?.blockReason; + if ( + typeof blockReason === "string" && + blockReason.length > 0 && + blockReason !== "BLOCK_REASON_UNSPECIFIED" + ) { + nativeStopReason = blockReason; + const replay = replayForActive(); + if (replay !== undefined) yield replay; + active = undefined; + yield { + type: "error", + code: blockReason.toLowerCase(), + message: safeProviderMessage( + typeof promptFeedback?.blockReasonMessage === "string" + ? promptFeedback.blockReasonMessage + : `Google blocked the prompt with: ${blockReason}`, + options.secretValues, + ), + retryable: false, + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + + if (chunk.candidates !== undefined && !Array.isArray(chunk.candidates)) { + throw new GoogleGenerativeAiCodecError("Google candidates must be an array"); + } + const candidate = Array.isArray(chunk.candidates) ? object(chunk.candidates[0]) : undefined; + if (candidate !== undefined) { + const content = object(candidate.content); + if (content?.parts !== undefined && !Array.isArray(content.parts)) { + throw new GoogleGenerativeAiCodecError("Google candidate parts must be an array"); + } + for (const rawPart of Array.isArray(content?.parts) ? content.parts : []) { + const part = object(rawPart); + if (part === undefined) + throw new GoogleGenerativeAiCodecError("Google part must be an object"); + if (part.text !== undefined) { + if (typeof part.text !== "string") { + throw new GoogleGenerativeAiCodecError("Google text part is malformed"); + } + const type = part.thought === true ? "thinking" : "text"; + if (part.thoughtSignature !== undefined && typeof part.thoughtSignature !== "string") { + throw new GoogleGenerativeAiCodecError("Google thought signature is malformed"); + } + if (active?.type !== type) { + const replay = replayForActive(); + if (replay !== undefined) yield replay; + active = { type, contentIndex: nextContentIndex++ }; + } + if (typeof part.thoughtSignature === "string" && part.thoughtSignature.length > 0) { + active.signature = googleSignature(part.thoughtSignature, "Google thought signature"); + } + if (part.text.length > 0) { + emittedContent = true; + yield { + type: type === "thinking" ? "thinking_delta" : "text_delta", + text: part.text, + contentIndex: active.contentIndex, + }; + } + } + if (part.functionCall !== undefined) { + const replay = replayForActive(); + if (replay !== undefined) yield replay; + active = undefined; + const call = object(part.functionCall); + if ( + call === undefined || + typeof call.name !== "string" || + call.name.length === 0 || + (call.id !== undefined && typeof call.id !== "string") + ) { + throw new GoogleGenerativeAiCodecError("Google function call is malformed"); + } + const input = call.args === undefined ? {} : object(call.args); + if (input === undefined) { + throw new GoogleGenerativeAiCodecError( + "Google function call arguments must be an object", + ); + } + let callId = typeof call.id === "string" && call.id.length > 0 ? call.id : ""; + if (callId.length === 0 || callIds.has(callId)) { + generatedCallId += 1; + callId = `google_call_${generatedCallId}`; + while (callIds.has(callId)) { + generatedCallId += 1; + callId = `google_call_${generatedCallId}`; + } + } + callIds.add(callId); + const contentIndex = nextContentIndex++; + const name = reverseToolName(options.request, call.name); + const argumentsText = JSON.stringify(input); + emittedContent = true; + emittedToolCall = true; + yield { type: "tool_call_start", contentIndex, callId, name }; + if (argumentsText.length > 0) { + yield { type: "tool_call_delta", contentIndex, callId, argumentsDelta: argumentsText }; + } + yield { type: "tool_call", contentIndex, callId, name, input: input as JsonObject }; + if (typeof part.thoughtSignature === "string" && part.thoughtSignature.length > 0) { + yield { + type: "replay_metadata", + target: "tool_call", + contentIndex, + callId, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.request.modelId, + signature: googleSignature(part.thoughtSignature, "Google tool thought signature"), + }; + } else if (part.thoughtSignature !== undefined) { + throw new GoogleGenerativeAiCodecError("Google thought signature is malformed"); + } + } + } + + if (candidate.finishReason !== undefined) { + if (typeof candidate.finishReason !== "string" || candidate.finishReason.length === 0) { + throw new GoogleGenerativeAiCodecError("Google finish reason is malformed"); + } + nativeStopReason = candidate.finishReason; + const replay = replayForActive(); + if (replay !== undefined) yield replay; + active = undefined; + if (nativeStopReason === "STOP") { + yield { + type: "completed", + stopReason: emittedToolCall ? "tool_use" : "stop", + usage: finalUsage, + response: responseMetadata(), + }; + return; + } + if (nativeStopReason === "MAX_TOKENS") { + yield { + type: "completed", + stopReason: "length", + usage: finalUsage, + partial: true, + response: responseMetadata(), + }; + return; + } + const message = + typeof candidate.finishMessage === "string" + ? candidate.finishMessage + : `Provider stopped with: ${nativeStopReason}`; + yield { + type: "error", + code: SAFETY_FINISH_REASONS.has(nativeStopReason) + ? nativeStopReason.toLowerCase() + : "google_generation_failure", + message: safeProviderMessage(message, options.secretValues), + retryable: false, + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + } + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + } +} diff --git a/packages/ai/src/google-vertex.ts b/packages/ai/src/google-vertex.ts new file mode 100644 index 0000000..734a898 --- /dev/null +++ b/packages/ai/src/google-vertex.ts @@ -0,0 +1,241 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Google Vertex AI dialect, endpoint, and request authentication policy. + +import type { JsonObject } from "@axl/protocol"; + +import type { ModelInfo, ModelStreamEvent } from "./model.ts"; +import { + decodeGoogleStream, + type EncodedGoogleRequest, + encodeGoogleRequest, + type GoogleDecodeOptions, +} from "./google-shared.ts"; +import type { PreparedModelRequest } from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; +import { stripTrailingSlashes } from "./transport-safety.ts"; + +export const DEFAULT_GOOGLE_VERTEX_API_VERSION = "v1"; + +const GOOGLE_CLOUD_SCOPE = "https://www.googleapis.com/auth/cloud-platform"; +const API_VERSION_PATTERN = /^v\d+(?:beta\d+)?$/; +const RESOURCE_SEGMENT_PATTERN = /^[A-Za-z0-9._~-]+$/; +const MODEL_SEGMENT_PATTERN = /^[A-Za-z0-9._~@-]+$/; + +export class GoogleVertexCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "GoogleVertexCodecError"; + } +} + +export type GoogleVertexCredentialPolicy = + | { + readonly type: "api_key"; + readonly apiKey: string; + } + | { + readonly type: "adc"; + readonly accessToken: string; + } + | { + readonly type: "service_account"; + readonly accessToken: string; + /** Used by the later token acquisition layer and never sent to Vertex. */ + readonly credentialsFile: string; + }; + +export interface GoogleVertexRequestPolicy { + readonly credential: GoogleVertexCredentialPolicy; + readonly project?: string; + readonly location?: string; + /** Optional collection endpoint for a proxy or private service route. */ + readonly baseUrl?: string; + readonly apiVersion?: string; +} + +export interface EncodedGoogleVertexRequest { + readonly url: string; + readonly body: JsonObject; + readonly headers: Readonly>; +} + +export type GoogleVertexDecodeOptions = GoogleDecodeOptions; + +function nonEmpty(value: string | undefined, label: string): string { + const trimmed = value?.trim(); + if (!trimmed) throw new GoogleVertexCodecError(`Google Vertex ${label} is required`); + return trimmed; +} + +function safeHeaderValue(value: string, label: string): string { + if (/[\r\n]/.test(value)) { + throw new GoogleVertexCodecError(`Google Vertex ${label} contains invalid characters`); + } + return value; +} + +function resourceSegment(value: string | undefined, label: string): string { + const segment = nonEmpty(value, label); + if (!RESOURCE_SEGMENT_PATTERN.test(segment)) { + throw new GoogleVertexCodecError(`Google Vertex ${label} is invalid`); + } + return segment; +} + +function apiVersion(value: string | undefined): string { + const version = value?.trim() || DEFAULT_GOOGLE_VERTEX_API_VERSION; + if (!API_VERSION_PATTERN.test(version)) { + throw new GoogleVertexCodecError("Google Vertex API version is invalid"); + } + return version; +} + +function modelResource(modelId: string): string { + const id = nonEmpty(modelId, "model ID"); + if (id.includes("..") || /[?#&]/.test(id)) { + throw new GoogleVertexCodecError("Google Vertex model ID is invalid"); + } + const segments = id.split("/"); + if (segments.some((segment) => !MODEL_SEGMENT_PATTERN.test(segment))) { + throw new GoogleVertexCodecError("Google Vertex model ID is invalid"); + } + if (id.startsWith("projects/") || id.startsWith("publishers/")) return id; + if (segments.length === 1) return `publishers/google/models/${id}`; + if (segments.length === 2) return `publishers/${segments[0]}/models/${segments[1]}`; + throw new GoogleVertexCodecError("Google Vertex model ID has an unsupported resource shape"); +} + +function parseBaseUrl(value: string): URL { + let url: URL; + try { + url = new URL(value.trim()); + } catch (cause) { + throw new GoogleVertexCodecError("Google Vertex base URL is invalid", { cause }); + } + if (url.protocol !== "https:" && url.protocol !== "http:") { + throw new GoogleVertexCodecError("Google Vertex base URL must use HTTP or HTTPS"); + } + if (url.username || url.password || url.hash) { + throw new GoogleVertexCodecError("Google Vertex base URL contains unsupported URL data"); + } + url.pathname = stripTrailingSlashes(url.pathname); + return url; +} + +function credentialType( + credential: GoogleVertexCredentialPolicy, +): GoogleVertexCredentialPolicy["type"] { + if ( + typeof credential !== "object" || + credential === null || + (credential.type !== "api_key" && + credential.type !== "adc" && + credential.type !== "service_account") + ) { + throw new GoogleVertexCodecError("Google Vertex credential policy is invalid"); + } + return credential.type; +} + +function standardBaseUrl(location: string, version: string): URL { + const host = + location === "global" + ? "aiplatform.googleapis.com" + : location === "us" || location === "eu" + ? `aiplatform.${location}.rep.googleapis.com` + : `${location}-aiplatform.googleapis.com`; + return new URL(`https://${host}/${version}`); +} + +function customBaseUrl(value: string, version: string): URL { + const url = parseBaseUrl(value); + const segments = url.pathname.split("/").filter(Boolean); + if (!segments.some((segment) => API_VERSION_PATTERN.test(segment))) { + url.pathname = `${url.pathname}/${version}`; + } + return url; +} + +function appendResource(url: URL, resource: string): URL { + const result = new URL(url); + result.pathname = `${stripTrailingSlashes(result.pathname)}/${resource}:streamGenerateContent`; + result.searchParams.set("alt", "sse"); + return result; +} + +/** + * Resolves the Vertex collection URL without acquiring credentials. API keys + * select Express Mode. ADC and service-account tokens require project and + * location and use the regional resource path. A custom URL is already a + * collection URL, matching the Google SDK collection-scope behavior. + */ +export function googleVertexStreamUrl(modelId: string, policy: GoogleVertexRequestPolicy): string { + const version = apiVersion(policy.apiVersion); + const resource = modelResource(modelId); + const type = credentialType(policy.credential); + if (type === "api_key") { + const base = + policy.baseUrl === undefined + ? new URL(`https://aiplatform.googleapis.com/${version}`) + : customBaseUrl(policy.baseUrl, version); + return appendResource(base, resource).toString(); + } + + const project = resourceSegment(policy.project, "project"); + const location = resourceSegment(policy.location, "location"); + if (policy.baseUrl !== undefined) { + return appendResource(customBaseUrl(policy.baseUrl, version), resource).toString(); + } + const base = standardBaseUrl(location, version); + const scopedResource = resource.startsWith("projects/") + ? resource + : `projects/${project}/locations/${location}/${resource}`; + return appendResource(base, scopedResource).toString(); +} + +function credentialHeaders( + credential: GoogleVertexCredentialPolicy, +): Readonly> { + credentialType(credential); + if (credential.type === "api_key") { + const key = safeHeaderValue(nonEmpty(credential.apiKey, "API key"), "API key"); + if (key === "gcp-vertex-credentials" || /^<[^>]+>$/.test(key)) { + throw new GoogleVertexCodecError("Google Vertex API key is a placeholder"); + } + return { "x-goog-api-key": key }; + } + if (credential.type === "service_account") { + nonEmpty(credential.credentialsFile, "service account credentials file"); + } + const token = safeHeaderValue(nonEmpty(credential.accessToken, "access token"), "access token"); + return { authorization: `Bearer ${token}` }; +} + +/** Cloud OAuth scope required when the ADC or service-account layer acquires a token. */ +export function googleVertexOAuthScope(): string { + return GOOGLE_CLOUD_SCOPE; +} + +/** Composes shared Google content with Vertex endpoint and credential policy. */ +export function encodeGoogleVertexRequest( + model: ModelInfo, + request: PreparedModelRequest, + policy: GoogleVertexRequestPolicy, +): EncodedGoogleVertexRequest { + const encoded: EncodedGoogleRequest = encodeGoogleRequest(model, request, "google-vertex"); + return { + url: googleVertexStreamUrl(encoded.modelId, policy), + body: encoded.body, + headers: { ...encoded.headers, ...credentialHeaders(policy.credential) }, + }; +} + +/** Decodes Vertex SSE through the shared Google response codec. */ +export function decodeGoogleVertexStream( + frames: AsyncIterable, + options: GoogleVertexDecodeOptions, +): AsyncGenerator { + return decodeGoogleStream(frames, options, "google-vertex"); +} diff --git a/packages/ai/src/groq.ts b/packages/ai/src/groq.ts new file mode 100644 index 0000000..279f926 --- /dev/null +++ b/packages/ai/src/groq.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const GROQ_PROVIDER_ID = "groq"; +export const GROQ_API_KEY_ENV = "GROQ_API_KEY"; +export const GROQ_BASE_URL = "https://api.groq.com/openai/v1"; + +export const GROQ_PROVIDER_DEFINITION = { + id: GROQ_PROVIDER_ID, + displayName: "Groq", + apiKeyDisplayName: "Groq API key", + environmentVariables: [GROQ_API_KEY_ENV], + baseUrl: GROQ_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createGroqProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(GROQ_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/http-sse-provider.ts b/packages/ai/src/http-sse-provider.ts new file mode 100644 index 0000000..ca8a479 --- /dev/null +++ b/packages/ai/src/http-sse-provider.ts @@ -0,0 +1,363 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { ProviderAuthentication, ResolvedAuth } from "./auth.ts"; +import { AuthError } from "./auth.ts"; +import { safeProviderMessage } from "./diagnostics.ts"; +import type { + AuthMethod, + ModelErrorCategory, + ModelInfo, + ModelRequest, + ModelStreamEvent, +} from "./model.ts"; +import type { ModelProvider } from "./provider.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + prepareModelRequest, +} from "./request-preparation.ts"; +import { registerResolvedSecrets } from "./secret-context.ts"; +import { decodeSseStream, type SseFrame } from "./sse.ts"; +import { raceWithSignal, safeEndpoint, safeFetch } from "./transport-safety.ts"; + +const DEFAULT_TIMEOUT_MS = 120_000; +const DEFAULT_MAX_RETRIES = 2; +const MAX_RETRIES = 10; +const RETRYABLE = new Set([429, 500, 502, 503, 504]); + +export interface EncodedHttpSseRequest { + readonly url: string; + readonly body: unknown; + readonly headers?: Readonly>; +} + +export interface HttpSseCodec { + encode( + model: ModelInfo, + request: PreparedModelRequest, + resolved: ResolvedAuth, + ): EncodedHttpSseRequest; + decode( + frames: AsyncIterable, + options: HttpStreamDecodeOptions, + ): AsyncIterable; + decodeBody?( + body: ReadableStream, + options: HttpStreamDecodeOptions, + ): AsyncIterable; +} + +export interface HttpStreamDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs: number; + readonly now: () => number; + readonly secretValues: readonly string[]; +} + +export interface HttpSseProviderOptions { + readonly allowLoopbackHttp?: boolean; + readonly id: string; + readonly displayName: string; + readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; + readonly models: readonly ModelInfo[]; + readonly resolveAuth: (signal: AbortSignal) => Promise; + readonly codecFor: (model: ModelInfo) => HttpSseCodec; + readonly validateEndpoint?: (url: URL, model: ModelInfo, resolved: ResolvedAuth) => void; + readonly fetch?: typeof fetch; + readonly now?: () => number; +} + +function category(status: number): ModelErrorCategory { + if (status === 401) return "authentication"; + if (status === 403) return "authorization"; + if (status === 408) return "timeout"; + if (status === 429) return "rate_limit"; + if (status >= 500) return "provider_internal"; + return "invalid_request"; +} + +function retryDelay(response: Response, attempt: number, maximum: number, now: number): number { + const raw = response.headers.get("retry-after")?.trim(); + const seconds = raw === undefined ? Number.NaN : Number(raw); + const date = raw === undefined ? Number.NaN : Date.parse(raw); + const advised = Number.isFinite(seconds) + ? Math.max(0, seconds * 1_000) + : Number.isFinite(date) + ? Math.max(0, date - now) + : 250 * 2 ** attempt; + return Math.min(advised, maximum); +} + +function wait(ms: number, signal: AbortSignal): Promise { + signal.throwIfAborted(); + return new Promise((resolve, reject) => { + const timer = setTimeout(done, ms); + function done(): void { + signal.removeEventListener("abort", abort); + resolve(); + } + function abort(): void { + clearTimeout(timer); + reject(signal.reason); + } + signal.addEventListener("abort", abort, { once: true }); + }); +} + +/** Shared finite HTTP and SSE lifecycle for native provider codecs. */ +export class HttpSseProvider implements ModelProvider { + readonly id: string; + readonly displayName: string; + readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; + private models: readonly ModelInfo[]; + private readonly resolveAuth: (signal: AbortSignal) => Promise; + private readonly codecFor: (model: ModelInfo) => HttpSseCodec; + private readonly validateEndpoint: + | ((url: URL, model: ModelInfo, resolved: ResolvedAuth) => void) + | undefined; + private readonly fetchImpl: typeof fetch | undefined; + private readonly now: () => number; + private readonly allowLoopbackHttp: boolean; + + constructor(options: HttpSseProviderOptions) { + this.id = options.id; + this.allowLoopbackHttp = options.allowLoopbackHttp ?? false; + this.displayName = options.displayName; + this.authMethods = [...options.authMethods]; + if (options.authentication !== undefined) this.authentication = options.authentication; + this.models = [...options.models]; + this.resolveAuth = options.resolveAuth; + this.codecFor = options.codecFor; + this.validateEndpoint = options.validateEndpoint; + this.fetchImpl = options.fetch; + this.now = options.now ?? Date.now; + } + + listModels(): Promise { + return Promise.resolve(this.models); + } + + replaceModels(models: readonly ModelInfo[]): void { + this.models = [...models]; + } + + stream(request: ModelRequest): AsyncIterable { + const model = this.models.find((candidate) => candidate.modelId === request.modelId); + if (model === undefined) + throw new TypeError(`Provider ${this.id} has no model ${request.modelId}`); + return this.run(model, request); + } + + streamModel(model: ModelInfo, request: ModelRequest): AsyncIterable { + if (model.providerId !== this.id || model.modelId !== request.modelId) { + throw new TypeError( + `Provider ${this.id} cannot dispatch foreign model ${model.providerId}/${model.modelId}`, + ); + } + return this.run(model, request); + } + + private async *run(model: ModelInfo, request: ModelRequest): AsyncGenerator { + const timeout = AbortSignal.timeout(request.timeoutMs ?? DEFAULT_TIMEOUT_MS); + const signal = + request.signal === undefined ? timeout : AbortSignal.any([request.signal, timeout]); + let prepared: PreparedModelRequest; + let encoded: EncodedHttpSseRequest; + let resolved: ResolvedAuth; + let secrets: readonly string[] = []; + let codec: HttpSseCodec; + try { + prepared = isPreparedModelRequest(request) + ? request + : await prepareModelRequest(model, request); + resolved = await raceWithSignal(this.resolveAuth(signal), signal); + secrets = resolved.secretValues; + registerResolvedSecrets(secrets); + codec = this.codecFor(model); + encoded = codec.encode(model, prepared, resolved); + const requestUrl = new URL( + safeEndpoint(encoded.url, { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: this.allowLoopbackHttp, + allowQuery: true, + }), + ); + this.validateEndpoint?.(requestUrl, model, resolved); + } catch (error) { + yield this.failure( + request, + signal, + error, + secrets, + "provider_request_setup_failed", + "before_dispatch", + ); + return; + } + + const maximumRetries = Math.min(request.maxRetries ?? DEFAULT_MAX_RETRIES, MAX_RETRIES); + const maximumDelay = request.maxRetryDelayMs ?? 30_000; + let response: Response | undefined; + const body = JSON.stringify(encoded.body); + for (let attempt = 0; attempt <= maximumRetries; attempt += 1) { + try { + const unsignedHeaders = { + accept: "text/event-stream", + "content-type": "application/json", + ...encoded.headers, + }; + const headers = + resolved.auth.signRequest === undefined + ? unsignedHeaders + : await raceWithSignal( + resolved.auth.signRequest( + { method: "POST", url: encoded.url, headers: unsignedHeaders, body }, + signal, + ), + signal, + ); + signal.throwIfAborted(); + response = await raceWithSignal( + safeFetch( + encoded.url, + { + method: "POST", + headers, + body, + signal, + }, + { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: this.allowLoopbackHttp, + expectedOrigin: new URL(encoded.url).origin, + ...(this.fetchImpl === undefined ? {} : { fetch: this.fetchImpl }), + }, + ), + signal, + ); + } catch (error) { + yield this.failure( + request, + signal, + error, + secrets, + "provider_request_failed", + "before_dispatch", + ); + return; + } + if (response.ok) break; + const retryable = RETRYABLE.has(response.status); + if (retryable && attempt < maximumRetries) { + const delay = retryDelay(response, attempt, maximumDelay, this.now()); + await response.body?.cancel(); + try { + await wait(delay, signal); + } catch (error) { + yield this.failure( + request, + signal, + error, + secrets, + "provider_request_failed", + "before_dispatch", + ); + return; + } + continue; + } + await response.body?.cancel(); + yield { + type: "error", + code: `http_${response.status}`, + message: `Provider ${this.id} returned ${response.status}`, + retryable, + category: category(response.status), + requestPhase: "awaiting_response", + }; + return; + } + if (response?.body == null) { + yield { + type: "error", + code: "empty_response", + message: `Provider ${this.id} returned no response body`, + retryable: false, + category: "provider_internal", + requestPhase: "awaiting_response", + }; + return; + } + let partial = false; + try { + const decodeOptions = { + model, + request: prepared, + startedAtMs: this.now(), + now: this.now, + secretValues: secrets, + }; + const events = + codec.decodeBody?.(response.body, decodeOptions) ?? + codec.decode(decodeSseStream(response.body), decodeOptions); + const iterator = events[Symbol.asyncIterator](); + for (;;) { + const next = await raceWithSignal(iterator.next(), signal); + if (next.done) break; + const event = next.value; + if (!new Set(["completed", "error", "aborted"]).has(event.type)) partial = true; + yield event; + if (new Set(["completed", "error", "aborted"]).has(event.type)) return; + } + } catch (error) { + yield this.failure( + request, + signal, + error, + secrets, + "provider_stream_failed", + "streaming", + partial, + ); + } + } + + private failure( + request: ModelRequest, + operationSignal: AbortSignal, + error: unknown, + secrets: readonly string[], + code: string, + phase: "before_dispatch" | "streaming", + partial = false, + ): ModelStreamEvent { + if (request.signal?.aborted) return { type: "aborted", ...(partial ? { partial: true } : {}) }; + if (operationSignal.aborted) { + return { + type: "error", + code: "provider_timeout", + message: `Provider ${this.id} request timed out`, + retryable: true, + category: "timeout", + requestPhase: phase, + ...(partial ? { partial: true } : {}), + }; + } + return { + type: "error", + code, + message: safeProviderMessage( + error instanceof Error ? error.message : "provider request failed", + secrets, + ), + retryable: false, + category: error instanceof AuthError ? "authentication" : "unknown", + requestPhase: phase, + ...(partial ? { partial: true } : {}), + }; + } +} diff --git a/packages/ai/src/huggingface.ts b/packages/ai/src/huggingface.ts new file mode 100644 index 0000000..7c92ac1 --- /dev/null +++ b/packages/ai/src/huggingface.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const HUGGINGFACE_PROVIDER_ID = "huggingface"; +export const HUGGINGFACE_API_KEY_ENV = "HF_TOKEN"; +export const HUGGINGFACE_BASE_URL = "https://router.huggingface.co/v1"; + +export const HUGGINGFACE_PROVIDER_DEFINITION = { + id: HUGGINGFACE_PROVIDER_ID, + displayName: "Hugging Face", + apiKeyDisplayName: "Hugging Face token", + environmentVariables: [HUGGINGFACE_API_KEY_ENV], + baseUrl: HUGGINGFACE_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createHuggingFaceProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(HUGGINGFACE_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/index.ts b/packages/ai/src/index.ts index 39b0245..d6193f2 100644 --- a/packages/ai/src/index.ts +++ b/packages/ai/src/index.ts @@ -1,19 +1,68 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan // SPDX-License-Identifier: Apache-2.0 +export * from "./ant-ling.ts"; +export * from "./anthropic-messages.ts"; +export * from "./api-key-auth.ts"; export * from "./auth.ts"; +export * from "./aws-auth.ts"; +export * from "./aws-event-stream.ts"; export * from "./azure-openai.ts"; +export * from "./baseten.ts"; +export * from "./bedrock-converse-stream.ts"; +export * from "./builtin-providers.ts"; export * from "./capabilities.ts"; +export * from "./catalog.ts"; +export * from "./catalog-store.ts"; +export * from "./cerebras.ts"; +export * from "./cloud-auth.ts"; export * from "./credentials.ts"; +export * from "./deepseek.ts"; +export * from "./diagnostics.ts"; export * from "./dialect.ts"; export * from "./fake-provider.ts"; +export * from "./fireworks.ts"; +export * from "./gateway-messages.ts"; +export * from "./google-generative-ai.ts"; +export * from "./google-vertex.ts"; +export * from "./groq.ts"; +export * from "./http-sse-provider.ts"; +export * from "./huggingface.ts"; +export * from "./minimax.ts"; +export * from "./minimax-cn.ts"; +export * from "./mistral-conversations.ts"; export * from "./model.ts"; +export * from "./models-config.ts"; +export * from "./moonshotai.ts"; +export * from "./moonshotai-cn.ts"; +export * from "./nvidia.ts"; +export * from "./oauth-auth.ts"; +export * from "./openai-chat.ts"; +export * from "./openai-chat-provider.ts"; +export * from "./openai-codex-responses.ts"; export * from "./openai-responses.ts"; +export * from "./openrouter-images.ts"; export * from "./provider.ts"; export * from "./provider-port.ts"; +export * from "./qwen-token-plan.ts"; +export * from "./qwen-token-plan-cn.ts"; +export * from "./qwen-token-plan-individual.ts"; export * from "./registry.ts"; +export * from "./remaining-providers.ts"; +export * from "./request-configuration.ts"; +export * from "./request-preparation.ts"; export * from "./sse.ts"; +export * from "./static-openai-chat-provider.ts"; export * from "./stream.ts"; export * from "./thinking.ts"; +export * from "./together.ts"; +export * from "./transport-safety.ts"; export * from "./usage.ts"; -export * from "./request-configuration.ts"; +export * from "./vercel-ai-gateway.ts"; +export * from "./xai.ts"; +export * from "./xiaomi.ts"; +export * from "./xiaomi-token-plan-ams.ts"; +export * from "./xiaomi-token-plan-cn.ts"; +export * from "./xiaomi-token-plan-sgp.ts"; +export * from "./zai.ts"; +export * from "./zai-coding-cn.ts"; diff --git a/packages/ai/src/minimax-cn.ts b/packages/ai/src/minimax-cn.ts new file mode 100644 index 0000000..fd163e2 --- /dev/null +++ b/packages/ai/src/minimax-cn.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const MINIMAX_CN_PROVIDER_ID = "minimax-cn"; +export const MINIMAX_CN_API_KEY_ENV = "MINIMAX_CN_API_KEY"; +export const MINIMAX_CN_BASE_URL = "https://api.minimaxi.com/v1"; + +export const MINIMAX_CN_PROVIDER_DEFINITION = { + id: MINIMAX_CN_PROVIDER_ID, + displayName: "MiniMax China", + apiKeyDisplayName: "MiniMax China API key", + environmentVariables: [MINIMAX_CN_API_KEY_ENV], + baseUrl: MINIMAX_CN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createMiniMaxCnProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(MINIMAX_CN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/minimax.ts b/packages/ai/src/minimax.ts new file mode 100644 index 0000000..d016882 --- /dev/null +++ b/packages/ai/src/minimax.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const MINIMAX_PROVIDER_ID = "minimax"; +export const MINIMAX_API_KEY_ENV = "MINIMAX_API_KEY"; +export const MINIMAX_BASE_URL = "https://api.minimax.io/v1"; + +export const MINIMAX_PROVIDER_DEFINITION = { + id: MINIMAX_PROVIDER_ID, + displayName: "MiniMax", + apiKeyDisplayName: "MiniMax API key", + environmentVariables: [MINIMAX_API_KEY_ENV], + baseUrl: MINIMAX_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createMiniMaxProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(MINIMAX_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/mistral-conversations.ts b/packages/ai/src/mistral-conversations.ts new file mode 100644 index 0000000..9a8f9fc --- /dev/null +++ b/packages/ai/src/mistral-conversations.ts @@ -0,0 +1,646 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Transport-neutral Mistral Conversations request and SSE codec. + +import { createHash } from "node:crypto"; + +import type { JsonObject, JsonValue, Usage } from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { MistralCompatibility, ModelInfo, ModelStreamEvent } from "./model.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + type PreparedRequestMessage, + preparedBlobDataUrl, +} from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; +import { withUsageCost } from "./usage.ts"; + +const RESERVED_REQUEST_FIELDS = new Set([ + "model", + "stream", + "messages", + "tools", + "tool_choice", + "max_tokens", + "prompt_mode", + "reasoning_effort", + "prompt_cache_key", + "temperature", + "top_p", + "frequency_penalty", + "presence_penalty", + "random_seed", +]); + +export class MistralConversationsCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "MistralConversationsCodecError"; + } +} + +export interface EncodedMistralConversationsRequest { + readonly body: JsonObject; + /** Safe affinity headers only. Authentication remains transport-owned. */ + readonly headers: Readonly>; +} + +export interface MistralConversationsDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +type MutableJsonObject = Record; + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function compatibility(model: ModelInfo): MistralCompatibility { + if (model.apiDialect !== "mistral-conversations") { + throw new MistralConversationsCodecError( + `Model ${model.modelId} does not use the mistral-conversations dialect`, + ); + } + if (model.compatibility?.dialect !== "mistral-conversations") { + throw new MistralConversationsCodecError( + `Model ${model.modelId} has no Mistral Conversations compatibility record`, + ); + } + return model.compatibility; +} + +function requirePrepared(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new MistralConversationsCodecError( + "Mistral Conversations requires a prepared model request", + ); + } +} + +function contentParts( + request: PreparedModelRequest, + message: Extract, +): JsonValue[] { + return message.content.map((part) => { + if (part.type === "text") return { type: "text", text: part.text }; + const blob = request.preparation.blobs.get(part.blob.sha256); + if (blob === undefined) { + throw new MistralConversationsCodecError(`Prepared blob ${part.blob.sha256} is unavailable`); + } + return { type: "image_url", image_url: preparedBlobDataUrl(blob) }; + }); +} + +function rejectReplayMetadata( + message: Extract, + messageIndex: number, +): void { + if (message.continuation !== undefined) { + throw new MistralConversationsCodecError( + `messages[${messageIndex}] has continuation metadata unsupported by Mistral`, + ); + } + for (const [contentIndex, part] of message.content.entries()) { + if (part.type === "text" && part.continuation !== undefined) { + throw new MistralConversationsCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported continuation metadata`, + ); + } + if ((part.type === "text" || part.type === "thinking") && part.signature !== undefined) { + throw new MistralConversationsCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported replay signature`, + ); + } + } + for (const [callIndex, call] of (message.toolCalls ?? []).entries()) { + if (call.signature !== undefined || call.continuation !== undefined) { + throw new MistralConversationsCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported replay metadata`, + ); + } + } +} + +function assistantMessage( + message: Extract, + messageIndex: number, +): JsonObject { + rejectReplayMetadata(message, messageIndex); + const content: JsonValue[] = []; + for (const [contentIndex, part] of message.content.entries()) { + if (part.type === "blob") { + throw new MistralConversationsCodecError( + `messages[${messageIndex}].content[${contentIndex}] cannot replay an assistant image`, + ); + } + if (part.text.trim().length === 0) continue; + content.push( + part.type === "thinking" + ? { type: "thinking", thinking: [{ type: "text", text: part.text }] } + : { type: "text", text: part.text }, + ); + } + const toolCalls = message.toolCalls?.map( + (call, index): JsonValue => ({ + id: call.callId, + type: "function", + function: { name: call.name, arguments: JSON.stringify(call.input) }, + index, + }), + ); + if (content.length === 0 && (toolCalls === undefined || toolCalls.length === 0)) { + throw new MistralConversationsCodecError( + `messages[${messageIndex}] has no Mistral-renderable assistant content`, + ); + } + return { + role: "assistant", + prefix: false, + ...(content.length === 0 ? {} : { content }), + ...(toolCalls === undefined || toolCalls.length === 0 ? {} : { tool_calls: toolCalls }), + }; +} + +function toolResultContent( + request: PreparedModelRequest, + message: Extract, +): JsonValue[] { + const parts = contentParts(request, message); + const text = parts + .filter( + (part): part is { type: "text"; text: string } => + object(part)?.type === "text" && typeof object(part)?.text === "string", + ) + .map((part) => part.text) + .join("\n") + .trim(); + const images = parts.filter((part) => object(part)?.type === "image_url"); + let renderedText = text; + if (message.isError) renderedText = `[tool error] ${renderedText || "(no tool output)"}`; + else if (!renderedText) + renderedText = images.length > 0 ? "(see attached image)" : "(no tool output)"; + return [{ type: "text", text: renderedText }, ...images]; +} + +function encodeMessages(model: ModelInfo, request: PreparedModelRequest): JsonValue[] { + const messages: JsonValue[] = []; + if (request.system !== undefined && request.system.length > 0) { + messages.push({ role: "system", content: request.system }); + } + for (const [messageIndex, message] of request.messages.entries()) { + if (message.role === "user") { + const parts = contentParts(request, message); + const only = parts[0]; + messages.push({ + role: "user", + content: + parts.length === 1 && object(only)?.type === "text" + ? (object(only)?.text as string) + : parts, + }); + } else if (message.role === "assistant") { + messages.push(assistantMessage(message, messageIndex)); + } else { + messages.push({ + role: "tool", + tool_call_id: message.callId, + name: message.name, + content: toolResultContent(request, message), + }); + } + } + compatibility(model); + return messages; +} + +function encodeTools(request: PreparedModelRequest): JsonValue[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + return request.tools.map((tool): JsonValue => { + if (tool.preparedConstraint?.type === "grammar") { + throw new MistralConversationsCodecError( + `Mistral cannot render grammar-constrained tool ${tool.canonicalName}`, + ); + } + return { + type: "function", + function: { + name: tool.name, + description: tool.description, + parameters: tool.inputSchema, + strict: + tool.preparedConstraint?.type === "json-schema" ? tool.preparedConstraint.strict : false, + }, + }; + }); +} + +function applyReasoning(body: MutableJsonObject, request: PreparedModelRequest): void { + const reasoning = request.preparation.reasoning; + if (reasoning === undefined) return; + if (reasoning.providerValue !== undefined) { + body.reasoning_effort = reasoning.providerValue; + } else if (reasoning.effective !== "off") { + body.prompt_mode = "reasoning"; + } +} + +function applySampling(body: MutableJsonObject, request: PreparedModelRequest): void { + const sampling = request.sampling; + if (sampling === undefined) return; + const fields = { + temperature: "temperature", + topP: "top_p", + frequencyPenalty: "frequency_penalty", + presencePenalty: "presence_penalty", + seed: "random_seed", + } as const; + for (const [source, target] of Object.entries(fields) as [keyof typeof fields, string][]) { + const value = sampling[source]; + if (value !== undefined) body[target] = value; + } + for (const [field, value] of Object.entries(sampling.custom ?? {})) { + if (RESERVED_REQUEST_FIELDS.has(field) || field in body) { + throw new MistralConversationsCodecError( + `Custom sampling field ${field} collides with a Mistral request field`, + ); + } + body[field] = value; + } +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeMistralConversationsRequest( + model: ModelInfo, + request: PreparedModelRequest, +): EncodedMistralConversationsRequest { + requirePrepared(request); + compatibility(model); + if (request.modelId !== model.modelId) { + throw new MistralConversationsCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (request.metadata !== undefined && Object.keys(request.metadata).length > 0) { + throw new MistralConversationsCodecError("Mistral cannot render request metadata"); + } + const body: MutableJsonObject = { + model: model.modelId, + stream: true, + messages: encodeMessages(model, request), + }; + if (request.maxOutputTokens !== undefined) body.max_tokens = request.maxOutputTokens; + const tools = request.toolChoice === "none" ? undefined : encodeTools(request); + if (tools !== undefined) body.tools = tools; + if (request.toolChoice !== undefined) { + if (request.toolChoice !== "none" && tools === undefined) { + throw new MistralConversationsCodecError( + `toolChoice ${request.toolChoice} needs at least one tool`, + ); + } + body.tool_choice = request.toolChoice; + } + applyReasoning(body, request); + applySampling(body, request); + const headers: Record = {}; + if (request.preparation.cache.retention !== "none") { + const sessionId = request.preparation.cache.sessionId; + if (sessionId !== undefined) { + body.prompt_cache_key = sessionId; + headers["x-affinity"] = sessionId; + } + } + return { body, headers }; +} + +interface ToolAccumulator { + readonly index: number; + readonly contentIndex: number; + id: string; + name: string; + argumentsText: string; + emittedArguments: number; + started: boolean; +} + +function nonNegative(value: unknown, label: string): number { + if (value === undefined || value === null) return 0; + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new MistralConversationsCodecError(`Mistral usage ${label} must be non-negative`); + } + return value; +} + +function usage(raw: unknown, model: ModelInfo): Usage { + const value = object(raw); + if (value === undefined) + throw new MistralConversationsCodecError("Mistral usage must be an object"); + const prompt = nonNegative(value.prompt_tokens, "prompt_tokens"); + const details = + object(value.prompt_tokens_details) ?? + object(value.prompt_token_details) ?? + object(value.promptTokensDetails) ?? + object(value.promptTokenDetails); + const cachedRaw = + details?.cached_tokens ?? + details?.cachedTokens ?? + value.num_cached_tokens ?? + value.numCachedTokens; + const cached = Math.min(prompt, nonNegative(cachedRaw, "cached_tokens")); + const mapped: Usage = { + inputTokens: prompt - cached, + outputTokens: nonNegative(value.completion_tokens, "completion_tokens"), + cacheReadTokens: cached, + cacheWriteTokens: 0, + reasoningTokens: 0, + }; + return model.cost === undefined ? mapped : withUsageCost(model.cost, mapped); +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function fallbackToolCallId(index: number): string { + return createHash("sha256").update(`mistral-tool:${index}`).digest("hex").slice(0, 9); +} + +function retryableCode(code: string): boolean { + return code === "rate_limit" || code === "rate_limit_exceeded" || code === "server_error"; +} + +/** Decodes native Mistral SSE frames into canonical model stream events. */ +export async function* decodeMistralConversationsStream( + frames: AsyncIterable, + options: MistralConversationsDecodeOptions, +): AsyncGenerator { + requirePrepared(options.request); + compatibility(options.model); + const tools = new Map(); + let nextContentIndex = 0; + let currentContent: { type: "text" | "thinking"; index: number } | undefined; + let finalUsage: Usage = { + inputTokens: 0, + outputTokens: 0, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + ...(options.model.cost === undefined ? {} : { costUsd: 0 }), + }; + let responseId: string | undefined; + let routedModelId: string | undefined; + let finishReason: string | undefined; + let emittedContent = false; + let emittedToolCall = false; + + const response = () => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined ? {} : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + ...(finishReason === undefined ? {} : { nativeStopReason: finishReason }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }); + + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (frame.data === "[DONE]") break; + let chunk: Record; + try { + const parsed = object(JSON.parse(frame.data) as unknown); + if (parsed === undefined) throw new Error("frame is not an object"); + chunk = parsed; + } catch (cause) { + throw new MistralConversationsCodecError("Mistral sent an undecodable stream frame", { + cause, + }); + } + + const providerError = object(chunk.error); + if (providerError !== undefined) { + const code = String(providerError.code ?? providerError.type ?? "provider_error"); + yield { + type: "error", + code, + message: safeProviderMessage( + typeof providerError.message === "string" + ? providerError.message + : "Mistral reported a failure", + options.secretValues, + ), + retryable: retryableCode(code), + category: + code === "rate_limit" || code === "rate_limit_exceeded" + ? "rate_limit" + : retryableCode(code) + ? "provider_internal" + : "invalid_request", + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: response(), + }; + return; + } + if (typeof chunk.id === "string" && chunk.id.length > 0) responseId ??= chunk.id; + if (typeof chunk.model === "string" && chunk.model.length > 0) routedModelId ??= chunk.model; + if (chunk.usage !== undefined) finalUsage = usage(chunk.usage, options.model); + if (!Array.isArray(chunk.choices)) { + throw new MistralConversationsCodecError("Mistral stream choices must be an array"); + } + if (chunk.choices.length === 0) continue; + if (chunk.choices.length !== 1) { + throw new MistralConversationsCodecError("Mistral returned multiple completion choices"); + } + const choice = object(chunk.choices[0]); + if (choice === undefined) + throw new MistralConversationsCodecError("Mistral choice is malformed"); + if (choice.finish_reason !== undefined && choice.finish_reason !== null) { + if (typeof choice.finish_reason !== "string" || choice.finish_reason.length === 0) { + throw new MistralConversationsCodecError("Mistral finish reason is malformed"); + } + finishReason = choice.finish_reason; + } + const delta = object(choice.delta); + if (delta === undefined) throw new MistralConversationsCodecError("Mistral delta is malformed"); + + if (delta.content !== undefined && delta.content !== null) { + const items = typeof delta.content === "string" ? [delta.content] : delta.content; + if (!Array.isArray(items)) { + throw new MistralConversationsCodecError("Mistral content delta is malformed"); + } + for (const item of items) { + let type: "text" | "thinking"; + let text: string; + if (typeof item === "string") { + type = "text"; + text = item; + } else { + const part = object(item); + if (part?.type === "text" && typeof part.text === "string") { + type = "text"; + text = part.text; + } else if (part?.type === "thinking" && Array.isArray(part.thinking)) { + type = "thinking"; + text = part.thinking + .map((entry) => object(entry)?.text) + .filter((entry): entry is string => typeof entry === "string") + .join(""); + if ( + part.thinking.some( + (entry) => object(entry) === undefined || typeof object(entry)?.text !== "string", + ) + ) { + throw new MistralConversationsCodecError("Mistral thinking delta is malformed"); + } + } else { + throw new MistralConversationsCodecError("Mistral content item is malformed"); + } + } + if (text.length === 0) continue; + if (currentContent?.type !== type) currentContent = { type, index: nextContentIndex++ }; + emittedContent = true; + yield { + type: type === "text" ? "text_delta" : "thinking_delta", + text, + contentIndex: currentContent.index, + }; + } + } + + if (delta.tool_calls === undefined || delta.tool_calls === null) continue; + if (!Array.isArray(delta.tool_calls)) { + throw new MistralConversationsCodecError("Mistral tool call deltas must be an array"); + } + currentContent = undefined; + for (const rawTool of delta.tool_calls) { + const tool = object(rawTool); + if (tool === undefined || !Number.isSafeInteger(tool.index) || (tool.index as number) < 0) { + throw new MistralConversationsCodecError("Mistral tool call has no valid index"); + } + const index = tool.index as number; + let state = tools.get(index); + if (state === undefined) { + state = { + index, + contentIndex: nextContentIndex++, + id: "", + name: "", + argumentsText: "", + emittedArguments: 0, + started: false, + }; + tools.set(index, state); + } + if (typeof tool.id === "string" && tool.id.length > 0 && tool.id !== "null") + state.id ||= tool.id; + const functionCall = object(tool.function); + if (functionCall === undefined) { + throw new MistralConversationsCodecError("Mistral tool function is malformed"); + } + if (typeof functionCall.name === "string" && functionCall.name.length > 0) { + state.name ||= functionCall.name; + } + if (typeof functionCall.arguments === "string") { + state.argumentsText += functionCall.arguments; + } else if (functionCall.arguments !== undefined) { + const argumentsObject = object(functionCall.arguments); + if (argumentsObject === undefined) { + throw new MistralConversationsCodecError("Mistral tool arguments are malformed"); + } + state.argumentsText += JSON.stringify(argumentsObject); + } + state.id ||= fallbackToolCallId(index); + if (!state.started && state.name.length > 0) { + state.started = true; + emittedContent = true; + yield { + type: "tool_call_start", + contentIndex: state.contentIndex, + callId: state.id, + name: reverseToolName(options.request, state.name), + }; + } + if (state.started && state.argumentsText.length > state.emittedArguments) { + const argumentsDelta = state.argumentsText.slice(state.emittedArguments); + state.emittedArguments = state.argumentsText.length; + yield { + type: "tool_call_delta", + contentIndex: state.contentIndex, + callId: state.id, + argumentsDelta, + }; + } + } + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (finishReason === undefined) return; + for (const state of [...tools.values()].sort((left, right) => left.index - right.index)) { + if (!state.started) + throw new MistralConversationsCodecError("Mistral ended an incomplete tool call"); + let input: JsonObject; + try { + const parsed = object(JSON.parse(state.argumentsText || "{}") as unknown); + if (parsed === undefined) throw new Error("arguments are not an object"); + input = parsed as JsonObject; + } catch (cause) { + throw new MistralConversationsCodecError( + `Mistral tool call ${state.id} has undecodable arguments`, + { cause }, + ); + } + emittedToolCall = true; + yield { + type: "tool_call", + contentIndex: state.contentIndex, + callId: state.id, + name: reverseToolName(options.request, state.name), + input, + }; + } + + const mapped = + finishReason === "stop" + ? "stop" + : finishReason === "length" || finishReason === "model_length" + ? "length" + : finishReason === "tool_calls" + ? "tool_use" + : "error"; + if (mapped === "error") { + yield { + type: "error", + code: finishReason, + message: safeProviderMessage(`Provider stopped with: ${finishReason}`, options.secretValues), + retryable: false, + category: "provider_internal", + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: response(), + }; + return; + } + yield { + type: "completed", + stopReason: mapped === "tool_use" || emittedToolCall ? "tool_use" : mapped, + usage: finalUsage, + ...(mapped === "length" ? { partial: true } : {}), + response: response(), + }; +} diff --git a/packages/ai/src/model.ts b/packages/ai/src/model.ts index 1e2368a..9db2363 100644 --- a/packages/ai/src/model.ts +++ b/packages/ai/src/model.ts @@ -2,20 +2,35 @@ // SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 -import type { BlobReference, ModelMessage, ThinkingLevel, ToolDeclaration } from "@axl/protocol"; +import type { + AssistantContent, + BlobReference, + JsonObject, + SafeProviderDiagnostic, + ThinkingLevel, + ToolCallRequest, + ToolDeclaration, + Usage, + UserContent, +} from "@axl/protocol"; // The canonical stream and message shapes live in @axl/protocol so the // kernel can consume them without depending on this package. Re-exported here // so provider code keeps one import surface. export type { + ModelErrorCategory, ModelMessage, + ModelRequestPhase, ModelStreamError, ModelStreamEvent, + ProviderResponseMetadata, + ProviderRetryGuidance, + SafeProviderDiagnostic, TerminalModelStreamEvent, ToolCallRequest, ToolDeclaration, } from "@axl/protocol"; -export { isTerminalModelStreamEvent } from "@axl/protocol"; +export { isTerminalModelStreamEvent, parseModelStreamEvent } from "@axl/protocol"; /** * How a provider can authenticate. Credential storage and lifecycle are a @@ -23,24 +38,219 @@ export { isTerminalModelStreamEvent } from "@axl/protocol"; */ export type AuthMethod = "environment" | "file" | "oauth" | "ambient" | "keyless"; -export interface ModelCost { +export type KnownApiDialect = + | "openai-chat" + | "openai-responses" + | "azure-openai-responses" + | "openai-codex-responses" + | "anthropic-messages" + | "google-generative-ai" + | "google-vertex" + | "bedrock-converse-stream" + | "mistral-conversations" + | "gateway-messages" + | "fake"; + +/** Known built-in dialects plus explicit extension dialect identities. */ +export type ApiDialect = KnownApiDialect | (string & {}); + +export type CacheRetention = "none" | "short" | "long"; + +export interface ModelCostRates { readonly inputUsdPerMTok: number; readonly outputUsdPerMTok: number; readonly cacheReadUsdPerMTok?: number; readonly cacheWriteUsdPerMTok?: number; } +export interface ModelCostTier extends ModelCostRates { + /** This request-wide tier applies when total input exceeds the threshold. */ + readonly inputTokensAbove: number; +} + +export interface ModelCost extends ModelCostRates { + /** Sorted request-wide price tiers. The highest matching threshold applies. */ + readonly tiers?: readonly ModelCostTier[]; +} + export interface ModelCapabilities { readonly toolUse: boolean; readonly structuredOutput: boolean; readonly imageInput: boolean; } +export interface ModelCachePolicy { + readonly supported: boolean; + readonly defaultRetention: CacheRetention; + readonly supportedRetentions: readonly CacheRetention[]; +} + +export type ModelAvailabilityStatus = "available" | "preview" | "deprecated" | "unavailable"; + +export interface ModelAvailability { + readonly status: ModelAvailabilityStatus; + readonly reason?: string; +} + +export interface EndpointVariable { + /** Non-secret logical variable used in the endpoint template. */ + readonly name: string; + /** Public configuration key from which the variable is read. */ + readonly setting: string; + readonly required: boolean; + readonly defaultValue?: string; +} + +export type EndpointPolicy = + | { readonly type: "fixed"; readonly baseUrl: string } + | { + readonly type: "configured"; + readonly baseUrlSetting: string; + readonly defaultBaseUrl?: string; + } + | { + readonly type: "template"; + readonly template: string; + readonly variables: readonly EndpointVariable[]; + }; + +export type ThinkingFormat = + | "openai" + | "openrouter" + | "deepseek" + | "together" + | "baseten" + | "zai" + | "qwen" + | "chat-template" + | "string-thinking" + | "ant-ling"; + +export type ThinkingTokenBudgetField = + | "thinking_token_budget" + | "thinking_budget" + | "thinking_budget_tokens"; + +export type SessionAffinityFormat = "openai" | "openai-no-session" | "openrouter"; + +export interface GatewayRoutingPolicy { + readonly only?: readonly string[]; + readonly order?: readonly string[]; + readonly ignore?: readonly string[]; + readonly allowFallbacks?: boolean; + readonly requireParameters?: boolean; + readonly dataCollection?: "allow" | "deny"; + readonly zeroDataRetention?: boolean; + readonly sort?: "price" | "throughput" | "latency"; +} + +export interface OpenAiChatCompatibility { + readonly dialect: "openai-chat"; + readonly supportsStore?: boolean; + readonly supportsDeveloperRole?: boolean; + readonly supportsReasoningEffort?: boolean; + readonly supportsUsageInStreaming?: boolean; + readonly supportsFinishReason?: boolean; + readonly maxTokensField?: "max_completion_tokens" | "max_tokens"; + readonly requiresToolResultName?: boolean; + readonly requiresAssistantAfterToolResult?: boolean; + readonly requiresThinkingAsText?: boolean; + readonly requiresReasoningContentOnAssistantMessages?: boolean; + readonly thinkingFormat?: ThinkingFormat; + readonly thinkingTokenBudgetField?: ThinkingTokenBudgetField; + readonly supportsGrammarTools?: boolean; + readonly supportsStrictTools?: boolean; + readonly cacheControlFormat?: "anthropic"; + readonly sessionAffinityFormat?: SessionAffinityFormat; + readonly supportsLongCacheRetention?: boolean; + readonly routing?: GatewayRoutingPolicy; +} + +export interface OpenAiResponsesCompatibility { + readonly dialect: "openai-responses" | "azure-openai-responses" | "openai-codex-responses"; + readonly supportsDeveloperRole?: boolean; + readonly supportsStrictTools?: boolean; + readonly supportsGrammarTools?: boolean; + readonly supportsLongCacheRetention?: boolean; + readonly supportsMaxOutputTokens?: boolean; + readonly sessionAffinityFormat?: SessionAffinityFormat; +} + +export interface AnthropicCompatibility { + readonly dialect: "anthropic-messages"; + readonly supportsLongCacheRetention?: boolean; + readonly supportsCacheControlOnTools?: boolean; + readonly supportsTemperature?: boolean; + readonly forceAdaptiveThinking?: boolean; + readonly allowEmptyThinkingSignature?: boolean; + readonly supportsStrictTools?: boolean; +} + +export interface GoogleGenerativeAiCompatibility { + readonly dialect: "google-generative-ai"; + readonly supportsStrictTools?: boolean; +} + +export interface GoogleVertexCompatibility { + readonly dialect: "google-vertex"; + readonly supportsStrictTools?: boolean; +} + +export interface BedrockCompatibility { + readonly dialect: "bedrock-converse-stream"; + readonly supportsStrictTools?: boolean; + readonly supportsPromptCacheMarkers?: boolean; + readonly supportsThinkingSignatures?: boolean; + readonly forceAdaptiveThinking?: boolean; +} + +export interface MistralCompatibility { + readonly dialect: "mistral-conversations"; + readonly supportsStrictTools?: boolean; +} + +export interface GatewayMessagesCompatibility { + readonly dialect: "gateway-messages"; + readonly supportsStrictTools?: boolean; +} + +export interface GenericCompatibility { + readonly dialect: "fake"; +} + +/** Dialect-specific compatibility controls used by built-in codecs. */ +export type ModelCompatibility = + | OpenAiChatCompatibility + | OpenAiResponsesCompatibility + | AnthropicCompatibility + | GoogleGenerativeAiCompatibility + | GoogleVertexCompatibility + | BedrockCompatibility + | MistralCompatibility + | GatewayMessagesCompatibility + | GenericCompatibility; + +export type SamplingOptionName = + | "temperature" + | "topP" + | "topK" + | "minP" + | "frequencyPenalty" + | "presencePenalty" + | "repetitionPenalty" + | "seed"; + +export interface ModelSamplingPolicy { + readonly supported: readonly SamplingOptionName[]; + /** Explicit allowlist for custom sampling keys on configured compatible endpoints. */ + readonly customFields?: readonly string[]; +} + export interface ModelInfo { readonly providerId: string; readonly modelId: string; readonly displayName: string; - /** The wire dialect the provider speaks for this model, e.g. `openai-chat`. */ + /** The wire dialect selected for this exact model. Legacy providers may declare their own. */ readonly apiDialect: string; readonly capabilities: ModelCapabilities; /** Whether the model can think at all. False means only the `off` level. */ @@ -54,24 +264,185 @@ export interface ModelInfo { readonly contextWindow: number; readonly maxOutputTokens: number; readonly cost?: ModelCost; - /** Extra headers the provider must send for this model. */ + readonly cache?: ModelCachePolicy; + readonly sampling?: ModelSamplingPolicy; + readonly endpoint?: EndpointPolicy; + readonly availability?: ModelAvailability; + /** Non-secret headers required by this model. Authentication headers are forbidden. */ readonly headers?: Readonly>; - /** Provider-specific compatibility flags, e.g. `{ strictJsonSchema: false }`. */ - readonly compatibility?: Readonly>; + /** Built-ins use typed controls; legacy providers may retain boolean compatibility metadata. */ + readonly compatibility?: ModelCompatibility | Readonly>; +} + +export interface ProviderModelIdentity { + readonly providerId: string; + readonly apiDialect: ApiDialect; + readonly modelId: string; +} + +/** Opaque replay data is usable only by the exact provider, dialect, and model that issued it. */ +export interface ProviderSignature extends ProviderModelIdentity { + readonly value: string; +} + +/** Provider continuation identifiers are provenance-bound and never treated as credentials. */ +export interface ProviderContinuationMetadata extends ProviderModelIdentity { + readonly responseId?: string; + readonly itemId?: string; + readonly namespace?: string; +} + +export type RequestAssistantContent = + | (Extract & { + readonly signature?: ProviderSignature; + readonly continuation?: ProviderContinuationMetadata; + }) + | (Extract & { + readonly signature?: ProviderSignature; + readonly redacted?: boolean; + }) + | Extract; + +export interface RequestToolCall extends ToolCallRequest { + readonly signature?: ProviderSignature; + readonly continuation?: ProviderContinuationMetadata; +} + +export type RequestModelMessage = + | { readonly role: "user"; readonly content: readonly UserContent[] } + | { + readonly role: "assistant"; + readonly content: readonly RequestAssistantContent[]; + readonly toolCalls?: readonly RequestToolCall[]; + readonly origin?: ProviderModelIdentity; + readonly continuation?: ProviderContinuationMetadata; + } + | { + readonly role: "tool"; + readonly callId: string; + readonly name: string; + readonly content: readonly UserContent[]; + readonly isError: boolean; + }; + +export type ToolConstraint = + | { readonly type: "json-schema"; readonly strict: "prefer" | "require" } + | { + readonly type: "grammar"; + readonly variants: { + readonly lark?: string; + readonly regex?: string; + }; + }; + +export interface RequestToolDeclaration extends ToolDeclaration { + readonly constraint?: ToolConstraint; +} + +export interface SamplingOptions { + readonly temperature?: number; + readonly topP?: number; + readonly topK?: number; + readonly minP?: number; + readonly frequencyPenalty?: number; + readonly presencePenalty?: number; + readonly repetitionPenalty?: number; + readonly seed?: number; + /** Explicit custom sampling fields for configured compatible endpoints. */ + readonly custom?: JsonObject; +} + +export interface CacheOptions { + readonly retention?: CacheRetention; + readonly sessionId?: string; +} + +export type ModelSafetyCategory = + | "HARM_CATEGORY_HARASSMENT" + | "HARM_CATEGORY_HATE_SPEECH" + | "HARM_CATEGORY_SEXUALLY_EXPLICIT" + | "HARM_CATEGORY_DANGEROUS_CONTENT" + | "HARM_CATEGORY_CIVIC_INTEGRITY"; + +export type ModelSafetyThreshold = + | "BLOCK_NONE" + | "BLOCK_LOW_AND_ABOVE" + | "BLOCK_MEDIUM_AND_ABOVE" + | "BLOCK_ONLY_HIGH" + | "OFF"; + +export interface ModelSafetySetting { + readonly category: ModelSafetyCategory; + readonly threshold: ModelSafetyThreshold; } -export interface ModelRequest { +export interface RequestControlOptions { + readonly timeoutMs?: number; + readonly maxRetries?: number; + readonly maxRetryDelayMs?: number; +} + +export interface ModelRequest extends RequestControlOptions { readonly modelId: string; readonly system?: string; - readonly messages: readonly ModelMessage[]; - readonly tools?: readonly ToolDeclaration[]; + readonly messages: readonly RequestModelMessage[]; + readonly tools?: readonly RequestToolDeclaration[]; readonly thinkingLevel?: ThinkingLevel; + readonly thinkingBudgets?: Readonly< + Partial> + >; readonly maxOutputTokens?: number; readonly httpIdleTimeoutMs?: number; readonly estimatedInputTokens?: number; readonly toolChoice?: "auto" | "required" | "none"; + readonly sampling?: SamplingOptions; + readonly cache?: CacheOptions; + /** Provider-neutral harm category thresholds, rendered only by supporting dialects. */ + readonly safetySettings?: readonly ModelSafetySetting[]; + /** Provider-safe request metadata. Credentials and authorization data are forbidden. */ + readonly metadata?: Readonly>; /** Resolves content-addressed media without placing bytes in canonical events. */ readonly readBlob?: (reference: BlobReference) => Promise; /** Cancellation for an in-flight stream travels through this signal. */ readonly signal?: AbortSignal; } + +export interface ImageModelInfo { + readonly providerId: string; + readonly modelId: string; + readonly displayName: string; + readonly apiDialect: "openrouter-images" | (string & {}); + readonly input: readonly ("text" | "image")[]; + readonly output: readonly ("text" | "image")[]; + readonly cost?: ModelCost; + readonly endpoint?: EndpointPolicy; + readonly availability?: ModelAvailability; +} + +export interface ImageGenerationRequest extends RequestControlOptions { + readonly modelId: string; + readonly prompt: string; + readonly inputImages?: readonly BlobReference[]; + readonly count?: number; + readonly size?: { readonly width: number; readonly height: number }; + readonly aspectRatio?: string; + readonly metadata?: Readonly>; + readonly readBlob?: (reference: BlobReference) => Promise; + /** Stores generated bytes outside events and returns their content-addressed references. */ + readonly writeBlob: ( + bytes: Uint8Array, + metadata: { readonly mediaType: string; readonly name?: string }, + ) => Promise; + readonly signal?: AbortSignal; +} + +export interface ImageGenerationResult { + readonly providerId: string; + readonly requestedModelId: string; + readonly routedModelId?: string; + readonly responseId?: string; + readonly images: readonly BlobReference[]; + readonly revisedPrompt?: string; + readonly usage?: Usage; + readonly diagnostics?: readonly SafeProviderDiagnostic[]; +} diff --git a/packages/ai/src/models-config.ts b/packages/ai/src/models-config.ts new file mode 100644 index 0000000..f9df9a2 --- /dev/null +++ b/packages/ai/src/models-config.ts @@ -0,0 +1,78 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import { access, readFile } from "node:fs/promises"; +import { join } from "node:path"; + +import { BUILTIN_PROVIDER_IDS } from "./builtin-providers.ts"; +import { object } from "./catalog-normalization.ts"; +import type { ModelProvider } from "./provider.ts"; +import { + createCustomProvider, + type ProviderFactoryOptions, + parseCustomProviderConfiguration, +} from "./remaining-providers.ts"; + +/** Native user configuration. Credentials remain in the provider-scoped secret store. */ +export async function loadConfiguredProviders( + axlHome: string, + options: ProviderFactoryOptions, +): Promise { + const path = join(axlHome, "models.json"); + try { + await access(join(axlHome, "custom-provider.json")); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; + return readConfiguration(path, options); + } + throw new Error( + `custom-provider.json has been replaced by ${path}; move its configuration under providers.custom and remove the old file`, + ); +} + +async function readConfiguration( + path: string, + options: ProviderFactoryOptions, +): Promise { + let text: string; + try { + text = await readFile(path, "utf8"); + } catch (error) { + if ((error as NodeJS.ErrnoException).code === "ENOENT") return []; + throw error; + } + try { + if (Buffer.byteLength(text) > 4 * 1024 * 1024) throw new Error("Configuration exceeds 4 MiB"); + const input = object(JSON.parse(text), "models.json"); + if (Object.keys(input).some((key) => key !== "providers")) { + throw new Error("models.json contains an unknown field"); + } + return Object.entries(object(input.providers, "providers")).map(([id, value]) => { + if (!/^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(id) || id.length > 128) { + throw new Error("Invalid configured provider ID"); + } + if (id !== "custom" && (BUILTIN_PROVIDER_IDS as readonly string[]).includes(id)) { + throw new Error(`Configured provider ${id} conflicts with a built-in provider`); + } + const entry = object(value, id); + const { displayName, ...configuration } = entry; + if ( + displayName !== undefined && + (typeof displayName !== "string" || + !displayName.trim() || + /[\p{Cc}\p{Cf}]/u.test(displayName)) + ) { + throw new Error(`Invalid display name for ${id}`); + } + return createCustomProvider({ + ...options, + ...parseCustomProviderConfiguration(configuration, id), + id, + ...(typeof displayName === "string" ? { displayName } : {}), + }); + }); + } catch (cause) { + // Do not echo config values: a malformed file may contain misplaced credentials. + throw new Error(`Invalid model configuration in ${path}`, { cause }); + } +} diff --git a/packages/ai/src/models.ts b/packages/ai/src/models.ts index 874e392..366a5a6 100644 --- a/packages/ai/src/models.ts +++ b/packages/ai/src/models.ts @@ -3,6 +3,15 @@ // SPDX-License-Identifier: Apache-2.0 export { AZURE_OPENAI_MODELS } from "./azure-openai-models.ts"; +export { + GENERATED_CATALOG_PROVENANCE, + getStaticModelCatalog, + listBuiltinCatalogProviders, + ModelCatalogValidationError, + STATIC_MODEL_CATALOG, + validateModelCatalog, +} from "./catalog.ts"; +export type { BuiltinCatalogProvider, CatalogProvenance } from "./catalog.ts"; export type { ModelInfo } from "./model.ts"; export { clampThinkingLevel, diff --git a/packages/ai/src/moonshotai-cn.ts b/packages/ai/src/moonshotai-cn.ts new file mode 100644 index 0000000..72c4490 --- /dev/null +++ b/packages/ai/src/moonshotai-cn.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const MOONSHOTAI_CN_PROVIDER_ID = "moonshotai-cn"; +export const MOONSHOTAI_CN_API_KEY_ENV = "MOONSHOT_API_KEY"; +export const MOONSHOTAI_CN_BASE_URL = "https://api.moonshot.cn/v1"; + +export const MOONSHOTAI_CN_PROVIDER_DEFINITION = { + id: MOONSHOTAI_CN_PROVIDER_ID, + displayName: "Moonshot AI China", + apiKeyDisplayName: "Moonshot AI China API key", + environmentVariables: [MOONSHOTAI_CN_API_KEY_ENV], + baseUrl: MOONSHOTAI_CN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createMoonshotAiCnProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(MOONSHOTAI_CN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/moonshotai.ts b/packages/ai/src/moonshotai.ts new file mode 100644 index 0000000..5efb9b5 --- /dev/null +++ b/packages/ai/src/moonshotai.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const MOONSHOTAI_PROVIDER_ID = "moonshotai"; +export const MOONSHOTAI_API_KEY_ENV = "MOONSHOT_API_KEY"; +export const MOONSHOTAI_BASE_URL = "https://api.moonshot.ai/v1"; + +export const MOONSHOTAI_PROVIDER_DEFINITION = { + id: MOONSHOTAI_PROVIDER_ID, + displayName: "Moonshot AI", + apiKeyDisplayName: "Moonshot AI API key", + environmentVariables: [MOONSHOTAI_API_KEY_ENV], + baseUrl: MOONSHOTAI_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createMoonshotAiProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(MOONSHOTAI_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/nvidia.ts b/packages/ai/src/nvidia.ts new file mode 100644 index 0000000..19b5e3e --- /dev/null +++ b/packages/ai/src/nvidia.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const NVIDIA_PROVIDER_ID = "nvidia"; +export const NVIDIA_API_KEY_ENV = "NVIDIA_API_KEY"; +export const NVIDIA_BASE_URL = "https://integrate.api.nvidia.com/v1"; + +export const NVIDIA_PROVIDER_DEFINITION = { + id: NVIDIA_PROVIDER_ID, + displayName: "NVIDIA NIM", + apiKeyDisplayName: "NVIDIA API key", + environmentVariables: [NVIDIA_API_KEY_ENV], + baseUrl: NVIDIA_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createNvidiaProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(NVIDIA_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/oauth-auth.ts b/packages/ai/src/oauth-auth.ts new file mode 100644 index 0000000..19c0e9c --- /dev/null +++ b/packages/ai/src/oauth-auth.ts @@ -0,0 +1,1039 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createHash, randomBytes, randomUUID } from "node:crypto"; + +import type { ApiKeyAuthMethod, OAuthAuthMethod, ProviderAuthInteraction } from "./auth.ts"; +import type { OAuthCredential } from "./credentials.ts"; +import { + raceWithSignal, + readBoundedJson, + safeFetch, + stripTrailingSlashes, +} from "./transport-safety.ts"; + +export interface OAuthFactoryOptions { + readonly fetch?: typeof fetch; + readonly now?: () => number; + readonly sleep?: (milliseconds: number, signal: AbortSignal) => Promise; +} + +const DEVICE_GRANT = "urn:ietf:params:oauth:grant-type:device_code"; +const DEFAULT_INTERVAL_SECONDS = 5; +const MINIMUM_INTERVAL_MS = 1_000; +const REFRESH_SKEW_MS = 5 * 60 * 1_000; +const PERMANENT_EXPIRY = Number.MAX_SAFE_INTEGER; + +type Json = Record; + +type DeviceCode = { + readonly deviceCode: string; + readonly userCode: string; + readonly verificationUri: string; + readonly intervalSeconds: number; + readonly expiresInSeconds: number; +}; + +function object(value: unknown, operation: string): Json { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new Error(`${operation} returned malformed JSON`); + } + return value as Json; +} + +function requiredString(value: Json, field: string, operation: string): string { + const result = value[field]; + if (typeof result !== "string" || result.length === 0) { + throw new Error(`${operation} response omitted ${field}`); + } + return result; +} + +function positiveNumber(value: Json, field: string, operation: string): number { + const result = value[field]; + if (typeof result !== "number" || !Number.isFinite(result) || result <= 0) { + throw new Error(`${operation} response has invalid ${field}`); + } + return result; +} + +function trustedUrl(value: string, operation: string): string { + let url: URL; + try { + url = new URL(value); + } catch (cause) { + throw new Error(`${operation} returned an invalid URL`, { cause }); + } + if (url.protocol !== "https:" && url.hostname !== "localhost" && url.hostname !== "127.0.0.1") { + throw new Error(`${operation} returned an untrusted URL`); + } + if (url.username || url.password) throw new Error(`${operation} returned an untrusted URL`); + return url.toString(); +} + +async function jsonRequest( + fetchImpl: typeof fetch, + url: string, + init: RequestInit, + operation: string, +): Promise { + const signal = init.signal as AbortSignal | undefined; + const operationRequest = safeFetch(url, init, { + label: `${operation} endpoint`, + expectedOrigin: new URL(url).origin, + ...(fetchImpl === fetch ? {} : { fetch: fetchImpl }), + }); + const response = + signal === undefined ? await operationRequest : await raceWithSignal(operationRequest, signal); + let body: unknown; + try { + body = await readBoundedJson(response, undefined, signal); + } catch (cause) { + throw new Error(`${operation} returned invalid JSON with status ${response.status}`, { cause }); + } + const result = object(body, operation); + if (!response.ok) { + const code = typeof result.error === "string" ? ` (${result.error})` : ""; + throw new Error(`${operation} failed with status ${response.status}${code}`); + } + return result; +} + +function form(fields: Readonly>): string { + return new URLSearchParams(fields).toString(); +} + +function formRequest(fields: Readonly>, signal: AbortSignal): RequestInit { + return { + method: "POST", + headers: { accept: "application/json", "content-type": "application/x-www-form-urlencoded" }, + body: form(fields), + signal, + }; +} + +function jsonPost(value: Json, signal: AbortSignal): RequestInit { + return { + method: "POST", + headers: { accept: "application/json", "content-type": "application/json" }, + body: JSON.stringify(value), + signal, + }; +} + +function defaultSleep(milliseconds: number, signal: AbortSignal): Promise { + signal.throwIfAborted(); + return new Promise((resolve, reject) => { + const timer = setTimeout(done, milliseconds); + function done(): void { + signal.removeEventListener("abort", abort); + resolve(); + } + function abort(): void { + clearTimeout(timer); + reject(signal.reason); + } + signal.addEventListener("abort", abort, { once: true }); + }); +} + +async function pollDevice(input: { + readonly signal: AbortSignal; + readonly intervalSeconds?: number; + readonly expiresInSeconds: number; + readonly sleep: (milliseconds: number, signal: AbortSignal) => Promise; + readonly poll: () => Promise< + | { readonly status: "pending" } + | { readonly status: "slow_down"; readonly intervalSeconds?: number } + | { readonly status: "complete"; readonly value: Result } + | { readonly status: "failed"; readonly message: string } + >; +}): Promise { + const deadline = Date.now() + input.expiresInSeconds * 1_000; + let interval = Math.max( + MINIMUM_INTERVAL_MS, + Math.floor((input.intervalSeconds ?? DEFAULT_INTERVAL_SECONDS) * 1_000), + ); + await input.sleep(Math.min(interval, Math.max(0, deadline - Date.now())), input.signal); + while (Date.now() < deadline) { + input.signal.throwIfAborted(); + const result = await input.poll(); + if (result.status === "complete") return result.value; + if (result.status === "failed") throw new Error(result.message); + if (result.status === "slow_down") { + interval = + result.intervalSeconds === undefined + ? interval + 5_000 + : Math.max(MINIMUM_INTERVAL_MS, Math.floor(result.intervalSeconds * 1_000)); + } + await input.sleep(Math.min(interval, Math.max(0, deadline - Date.now())), input.signal); + } + throw new Error("OAuth device authorization timed out"); +} + +function pkce(): { verifier: string; challenge: string } { + const verifier = randomBytes(32).toString("base64url"); + const challenge = createHash("sha256").update(verifier).digest("base64url"); + return { verifier, challenge }; +} + +function authorizationCode(input: string, expectedState?: string): string { + const trimmed = input.trim(); + if (!trimmed) throw new Error("OAuth authorization code is required"); + let code: string | null = null; + let state: string | null = null; + try { + const url = new URL(trimmed); + code = url.searchParams.get("code"); + state = url.searchParams.get("state"); + } catch { + if (trimmed.includes("code=")) { + const params = new URLSearchParams(trimmed); + code = params.get("code"); + state = params.get("state"); + } else if (trimmed.includes("#")) { + [code, state] = trimmed.split("#", 2) as [string, string]; + } else { + code = trimmed; + } + } + if (expectedState !== undefined && state !== null && state !== expectedState) { + throw new Error("OAuth state did not match"); + } + if (!code) throw new Error("OAuth authorization code is required"); + return code; +} + +async function promptForCode( + interaction: ProviderAuthInteraction, + url: string, + redirectUri: string, +): Promise { + interaction.notify({ + type: "auth_url", + url, + instructions: "Complete sign in, then paste the authorization code or final redirect URL.", + }); + return interaction.prompt({ + type: "manual_code", + message: "Paste the authorization code or final redirect URL:", + placeholder: redirectUri, + signal: interaction.signal, + }); +} + +function oauthCredential( + value: Json, + now: () => number, + operation: string, + previousRefresh?: string, + metadata?: Readonly>, +): OAuthCredential { + const access = requiredString(value, "access_token", operation); + const refresh = + value.refresh_token === undefined && previousRefresh !== undefined + ? previousRefresh + : requiredString(value, "refresh_token", operation); + const expiresIn = + value.expires_in === undefined ? 3_600 : positiveNumber(value, "expires_in", operation); + return { + type: "oauth", + access, + refresh, + expiresAt: Math.max(now(), now() + expiresIn * 1_000 - REFRESH_SKEW_MS), + ...(metadata === undefined ? {} : { metadata }), + }; +} + +function deviceCode(value: Json, operation: string): DeviceCode { + const verification = requiredString(value, "verification_uri", operation); + const complete = + typeof value.verification_uri_complete === "string" + ? value.verification_uri_complete + : verification; + return { + deviceCode: requiredString(value, "device_code", operation), + userCode: requiredString(value, "user_code", operation), + verificationUri: trustedUrl(complete, operation), + intervalSeconds: + typeof value.interval === "number" && value.interval > 0 + ? value.interval + : DEFAULT_INTERVAL_SECONDS, + expiresInSeconds: positiveNumber(value, "expires_in", operation), + }; +} + +function notifyDevice(interaction: ProviderAuthInteraction, device: DeviceCode): void { + interaction.notify({ + type: "device_code", + userCode: device.userCode, + verificationUri: device.verificationUri, + intervalSeconds: device.intervalSeconds, + expiresInSeconds: device.expiresInSeconds, + }); +} + +function oauthPollingResult( + body: Json, + response: Response, + complete: () => OAuthCredential, +): + | { readonly status: "pending" } + | { readonly status: "slow_down"; readonly intervalSeconds?: number } + | { readonly status: "complete"; readonly value: OAuthCredential } + | { readonly status: "failed"; readonly message: string } { + if (response.ok) return { status: "complete", value: complete() }; + if (body.error === "authorization_pending") return { status: "pending" }; + if (body.error === "slow_down") { + return { + status: "slow_down", + ...(typeof body.interval === "number" ? { intervalSeconds: body.interval } : {}), + }; + } + if (body.error === "access_denied" || body.error === "authorization_denied") { + return { status: "failed", message: "OAuth device authorization was denied" }; + } + if (body.error === "expired_token") { + return { status: "failed", message: "OAuth device authorization expired" }; + } + return { + status: "failed", + message: `OAuth device token request failed with status ${response.status}`, + }; +} + +async function pollFormToken(input: { + readonly fetchImpl: typeof fetch; + readonly url: string; + readonly fields: Readonly>; + readonly device: DeviceCode; + readonly signal: AbortSignal; + readonly sleep: (milliseconds: number, signal: AbortSignal) => Promise; + readonly now: () => number; + readonly operation: string; +}): Promise { + return pollDevice({ + signal: input.signal, + intervalSeconds: input.device.intervalSeconds, + expiresInSeconds: input.device.expiresInSeconds, + sleep: input.sleep, + poll: async () => { + const response = await raceWithSignal( + safeFetch(input.url, formRequest(input.fields, input.signal), { + label: `${input.operation} endpoint`, + expectedOrigin: new URL(input.url).origin, + ...(input.fetchImpl === fetch ? {} : { fetch: input.fetchImpl }), + }), + input.signal, + ); + let body: Json; + try { + body = object(await readBoundedJson(response, undefined, input.signal), input.operation); + } catch { + return { + status: "failed" as const, + message: `${input.operation} returned invalid JSON with status ${response.status}`, + }; + } + return oauthPollingResult(body, response, () => + oauthCredential(body, input.now, input.operation), + ); + }, + }); +} + +export function createAnthropicOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const clientId = Buffer.from( + "OWQxYzI1MGEtZTYxYi00NGQ5LTg4ZWQtNTk0NGQxOTYyZjVl", + "base64", + ).toString(); + const tokenUrl = "https://platform.claude.com/v1/oauth/token"; + return { + displayName: "Anthropic subscription", + login: async (interaction) => { + const { verifier, challenge } = pkce(); + const state = randomUUID(); + const redirectUri = "http://localhost:53692/callback"; + const url = new URL("https://claude.ai/oauth/authorize"); + url.search = new URLSearchParams({ + code: "true", + client_id: clientId, + response_type: "code", + redirect_uri: redirectUri, + scope: + "org:create_api_key user:profile user:inference user:sessions:claude_code user:mcp_servers user:file_upload", + code_challenge: challenge, + code_challenge_method: "S256", + state, + }).toString(); + const code = authorizationCode( + await promptForCode(interaction, url.toString(), redirectUri), + state, + ); + const body = await jsonRequest( + fetchImpl, + tokenUrl, + jsonPost( + { + grant_type: "authorization_code", + client_id: clientId, + code, + state, + redirect_uri: redirectUri, + code_verifier: verifier, + }, + interaction.signal, + ), + "Anthropic token exchange", + ); + return oauthCredential(body, now, "Anthropic token exchange"); + }, + refresh: async (credential, signal) => + oauthCredential( + await jsonRequest( + fetchImpl, + tokenUrl, + jsonPost( + { + grant_type: "refresh_token", + client_id: clientId, + refresh_token: credential.refresh, + }, + signal, + ), + "Anthropic token refresh", + ), + now, + "Anthropic token refresh", + credential.refresh, + ), + toAuth: (credential) => ({ + headers: { + authorization: `Bearer ${credential.access}`, + "anthropic-beta": "claude-code-20250219,oauth-2025-04-20", + }, + }), + }; +} + +const OPENAI_CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"; +const OPENAI_TOKEN_URL = "https://auth.openai.com/oauth/token"; + +function codexCredential(value: Json, now: () => number, operation: string): OAuthCredential { + const credential = oauthCredential(value, now, operation); + const segments = credential.access.split("."); + if (segments.length !== 3) throw new Error(`${operation} returned an invalid access token`); + let payload: Json; + try { + payload = object(JSON.parse(Buffer.from(segments[1] ?? "", "base64url").toString()), operation); + } catch (cause) { + throw new Error(`${operation} returned an invalid access token`, { cause }); + } + const claim = object(payload["https://api.openai.com/auth"], operation); + const accountId = requiredString(claim, "chatgpt_account_id", operation); + return { ...credential, metadata: { accountId } }; +} + +export function createOpenAiCodexOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const sleep = options.sleep ?? defaultSleep; + const exchange = async ( + code: string, + verifier: string, + redirectUri: string, + signal: AbortSignal, + ) => + codexCredential( + await jsonRequest( + fetchImpl, + OPENAI_TOKEN_URL, + formRequest( + { + grant_type: "authorization_code", + client_id: OPENAI_CLIENT_ID, + code, + code_verifier: verifier, + redirect_uri: redirectUri, + }, + signal, + ), + "OpenAI Codex token exchange", + ), + now, + "OpenAI Codex token exchange", + ); + return { + displayName: "OpenAI ChatGPT subscription", + login: async (interaction) => { + const method = await interaction.prompt({ + type: "select", + message: "Select OpenAI Codex login method:", + options: [ + { id: "browser", label: "Browser login" }, + { id: "device_code", label: "Device code login" }, + ], + }); + if (method === "device_code") { + const start = await jsonRequest( + fetchImpl, + "https://auth.openai.com/api/accounts/deviceauth/usercode", + jsonPost({ client_id: OPENAI_CLIENT_ID }, interaction.signal), + "OpenAI Codex device authorization", + ); + const interval = + typeof start.interval === "string" ? Number(start.interval) : start.interval; + const device = { + deviceCode: requiredString(start, "device_auth_id", "OpenAI Codex device authorization"), + userCode: requiredString(start, "user_code", "OpenAI Codex device authorization"), + verificationUri: "https://auth.openai.com/codex/device", + intervalSeconds: + typeof interval === "number" && Number.isFinite(interval) && interval >= 0 + ? interval + : DEFAULT_INTERVAL_SECONDS, + expiresInSeconds: 15 * 60, + }; + notifyDevice(interaction, device); + const authorization = await pollDevice({ + signal: interaction.signal, + intervalSeconds: device.intervalSeconds, + expiresInSeconds: device.expiresInSeconds, + sleep, + poll: async () => { + const response = await raceWithSignal( + safeFetch( + "https://auth.openai.com/api/accounts/deviceauth/token", + jsonPost( + { device_auth_id: device.deviceCode, user_code: device.userCode }, + interaction.signal, + ), + { + label: "OpenAI Codex device token endpoint", + expectedOrigin: "https://auth.openai.com", + ...(fetchImpl === fetch ? {} : { fetch: fetchImpl }), + }, + ), + interaction.signal, + ); + if (response.status === 403 || response.status === 404) return { status: "pending" }; + const body = object( + await readBoundedJson(response, undefined, interaction.signal), + "OpenAI Codex device token", + ); + if (!response.ok) { + return { + status: "failed", + message: `OpenAI Codex device token failed with status ${response.status}`, + }; + } + return { + status: "complete", + value: { + code: requiredString(body, "authorization_code", "OpenAI Codex device token"), + verifier: requiredString(body, "code_verifier", "OpenAI Codex device token"), + }, + }; + }, + }); + return exchange( + authorization.code, + authorization.verifier, + "https://auth.openai.com/deviceauth/callback", + interaction.signal, + ); + } + if (method !== "browser") throw new Error(`Unknown OpenAI Codex login method: ${method}`); + const { verifier, challenge } = pkce(); + const state = randomUUID(); + const redirectUri = "http://localhost:1455/auth/callback"; + const url = new URL("https://auth.openai.com/oauth/authorize"); + url.search = new URLSearchParams({ + response_type: "code", + client_id: OPENAI_CLIENT_ID, + redirect_uri: redirectUri, + scope: "openid profile email offline_access", + code_challenge: challenge, + code_challenge_method: "S256", + state, + id_token_add_organizations: "true", + codex_cli_simplified_flow: "true", + originator: "axl", + }).toString(); + const code = authorizationCode( + await promptForCode(interaction, url.toString(), redirectUri), + state, + ); + return exchange(code, verifier, redirectUri, interaction.signal); + }, + refresh: async (credential, signal) => + codexCredential( + await jsonRequest( + fetchImpl, + OPENAI_TOKEN_URL, + formRequest( + { + grant_type: "refresh_token", + client_id: OPENAI_CLIENT_ID, + refresh_token: credential.refresh, + }, + signal, + ), + "OpenAI Codex token refresh", + ), + now, + "OpenAI Codex token refresh", + ), + toAuth: (credential) => ({ apiKey: credential.access }), + }; +} + +export function createOpenRouterOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + return { + displayName: "OpenRouter OAuth", + login: async (interaction) => { + const { verifier, challenge } = pkce(); + const callbackUrl = `http://127.0.0.1:1457/oauth/callback/${randomUUID()}`; + const url = new URL("https://openrouter.ai/auth"); + url.search = new URLSearchParams({ + callback_url: callbackUrl, + code_challenge: challenge, + code_challenge_method: "S256", + }).toString(); + const code = authorizationCode(await promptForCode(interaction, url.toString(), callbackUrl)); + const body = await jsonRequest( + fetchImpl, + "https://openrouter.ai/api/v1/auth/keys", + jsonPost( + { code, code_verifier: verifier, code_challenge_method: "S256" }, + interaction.signal, + ), + "OpenRouter key exchange", + ); + return { type: "api_key", key: requiredString(body, "key", "OpenRouter key exchange") }; + }, + refresh: async (credential) => credential, + toAuth: (credential) => ({ apiKey: credential.access }), + }; +} + +export function createKimiCodingOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const sleep = options.sleep ?? defaultSleep; + const host = "https://auth.kimi.com"; + return { + displayName: "Kimi For Coding subscription", + login: async (interaction) => { + const body = await jsonRequest( + fetchImpl, + `${host}/api/oauth/device_authorization`, + formRequest({ client_id: "17e5f671-d194-4dfb-9706-5516cb48c098" }, interaction.signal), + "Kimi device authorization", + ); + const device = deviceCode(body, "Kimi device authorization"); + notifyDevice(interaction, device); + return pollFormToken({ + fetchImpl, + url: `${host}/api/oauth/token`, + fields: { + client_id: "17e5f671-d194-4dfb-9706-5516cb48c098", + device_code: device.deviceCode, + grant_type: DEVICE_GRANT, + }, + device, + signal: interaction.signal, + sleep, + now, + operation: "Kimi device token", + }); + }, + refresh: async (credential, signal) => + oauthCredential( + await jsonRequest( + fetchImpl, + `${host}/api/oauth/token`, + formRequest( + { + client_id: "17e5f671-d194-4dfb-9706-5516cb48c098", + grant_type: "refresh_token", + refresh_token: credential.refresh, + }, + signal, + ), + "Kimi token refresh", + ), + now, + "Kimi token refresh", + credential.refresh, + ), + toAuth: (credential) => ({ apiKey: credential.access }), + }; +} + +export function createXaiOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const sleep = options.sleep ?? defaultSleep; + const clientId = "b1a00492-073a-47ea-816f-4c329264a828"; + const tokenUrl = "https://auth.x.ai/oauth2/token"; + return { + displayName: "xAI subscription", + login: async (interaction) => { + const body = await jsonRequest( + fetchImpl, + "https://auth.x.ai/oauth2/device/code", + formRequest( + { + client_id: clientId, + scope: "openid profile email offline_access grok-cli:access api:access", + referrer: "axl", + }, + interaction.signal, + ), + "xAI device authorization", + ); + const device = deviceCode(body, "xAI device authorization"); + notifyDevice(interaction, device); + return pollFormToken({ + fetchImpl, + url: tokenUrl, + fields: { grant_type: DEVICE_GRANT, client_id: clientId, device_code: device.deviceCode }, + device, + signal: interaction.signal, + sleep, + now, + operation: "xAI device token", + }); + }, + refresh: async (credential, signal) => + oauthCredential( + await jsonRequest( + fetchImpl, + tokenUrl, + formRequest( + { grant_type: "refresh_token", client_id: clientId, refresh_token: credential.refresh }, + signal, + ), + "xAI token refresh", + ), + now, + "xAI token refresh", + credential.refresh, + ), + toAuth: (credential) => ({ apiKey: credential.access }), + }; +} + +function normalizeGitHubDomain(value: string): string { + const trimmed = value.trim(); + if (!trimmed) return "github.com"; + const parsed = new URL(trimmed.includes("://") ? trimmed : `https://${trimmed}`); + if (parsed.protocol !== "https:" || parsed.username || parsed.password || parsed.port) { + throw new Error("GitHub Enterprise domain is invalid"); + } + return parsed.hostname; +} + +function copilotBaseUrl(token: string, domain: string): string { + const endpoint = /(?:^|;)proxy-ep=([^;]+)/.exec(token)?.[1]; + if (endpoint) { + const host = endpoint.replace(/^proxy\./, "api."); + return stripTrailingSlashes(trustedUrl(`https://${host}`, "GitHub Copilot token")); + } + return domain === "github.com" + ? "https://api.individual.githubcopilot.com" + : `https://copilot-api.${domain}`; +} + +export function createGitHubCopilotTokenAuth(options: OAuthFactoryOptions = {}): ApiKeyAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const headers = { + accept: "application/json", + "user-agent": "GitHubCopilotChat/0.35.0", + "editor-version": "vscode/1.107.0", + "editor-plugin-version": "copilot-chat/0.35.0", + "copilot-integration-id": "vscode-chat", + }; + return { + displayName: "GitHub Copilot token", + login: async (interaction) => ({ + type: "api_key", + key: await interaction.prompt({ type: "secret", message: "Enter GitHub access token:" }), + env: { + GITHUB_ENTERPRISE_URL: await interaction.prompt({ + type: "text", + message: "GitHub Enterprise domain, or blank for github.com:", + placeholder: "github.com", + }), + }, + }), + resolve: async ({ context, credential, signal }) => { + signal.throwIfAborted(); + const githubToken = credential?.key ?? context.env("COPILOT_GITHUB_TOKEN"); + if (!githubToken) return undefined; + const domain = normalizeGitHubDomain( + credential?.env?.GITHUB_ENTERPRISE_URL ?? + context.env("GITHUB_ENTERPRISE_URL") ?? + context.env("GH_HOST") ?? + "github.com", + ); + if (/(?:^|;)proxy-ep=([^;]+)/.test(githubToken)) { + return { + auth: { apiKey: githubToken, baseUrl: copilotBaseUrl(githubToken, domain) }, + source: credential?.key === undefined ? "COPILOT_GITHUB_TOKEN" : "stored credential", + secretValues: [githubToken], + }; + } + const body = await jsonRequest( + fetchImpl, + `https://api.${domain}/copilot_internal/v2/token`, + { headers: { ...headers, authorization: `Bearer ${githubToken}` }, signal }, + "GitHub Copilot token exchange", + ); + const access = requiredString(body, "token", "GitHub Copilot token exchange"); + positiveNumber(body, "expires_at", "GitHub Copilot token exchange"); + signal.throwIfAborted(); + return { + auth: { apiKey: access, baseUrl: copilotBaseUrl(access, domain) }, + source: credential?.key === undefined ? "COPILOT_GITHUB_TOKEN" : "stored credential", + secretValues: [githubToken, access], + }; + }, + }; +} + +export function createGitHubCopilotOAuth(options: OAuthFactoryOptions = {}): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const sleep = options.sleep ?? defaultSleep; + const clientId = Buffer.from("SXYxLmI1MDdhMDhjODdlY2ZlOTg=", "base64").toString(); + const headers = { + accept: "application/json", + "user-agent": "GitHubCopilotChat/0.35.0", + "editor-version": "vscode/1.107.0", + "editor-plugin-version": "copilot-chat/0.35.0", + "copilot-integration-id": "vscode-chat", + }; + const exchange = async (githubToken: string, domain: string, signal: AbortSignal) => { + const body = await jsonRequest( + fetchImpl, + `https://api.${domain}/copilot_internal/v2/token`, + { headers: { ...headers, authorization: `Bearer ${githubToken}` }, signal }, + "GitHub Copilot token exchange", + ); + const access = requiredString(body, "token", "GitHub Copilot token exchange"); + const expires = positiveNumber(body, "expires_at", "GitHub Copilot token exchange"); + return { + type: "oauth" as const, + access, + refresh: githubToken, + expiresAt: Math.max(now(), expires * 1_000 - REFRESH_SKEW_MS), + metadata: { domain, baseUrl: copilotBaseUrl(access, domain) }, + }; + }; + return { + displayName: "GitHub Copilot", + login: async (interaction) => { + const domain = normalizeGitHubDomain( + await interaction.prompt({ + type: "text", + message: "GitHub Enterprise domain, or blank for github.com:", + placeholder: "github.com", + }), + ); + const start = await jsonRequest( + fetchImpl, + `https://${domain}/login/device/code`, + { + ...formRequest({ client_id: clientId, scope: "read:user" }, interaction.signal), + headers: { + accept: "application/json", + "content-type": "application/x-www-form-urlencoded", + "user-agent": headers["user-agent"], + }, + }, + "GitHub device authorization", + ); + const device = deviceCode(start, "GitHub device authorization"); + notifyDevice(interaction, device); + const githubToken = await pollDevice({ + signal: interaction.signal, + intervalSeconds: device.intervalSeconds, + expiresInSeconds: device.expiresInSeconds, + sleep, + poll: async () => { + const response = await raceWithSignal( + safeFetch( + `https://${domain}/login/oauth/access_token`, + { + ...formRequest( + { client_id: clientId, device_code: device.deviceCode, grant_type: DEVICE_GRANT }, + interaction.signal, + ), + headers: { + accept: "application/json", + "content-type": "application/x-www-form-urlencoded", + "user-agent": headers["user-agent"], + }, + }, + { + label: "GitHub device token endpoint", + expectedOrigin: `https://${domain}`, + ...(fetchImpl === fetch ? {} : { fetch: fetchImpl }), + }, + ), + interaction.signal, + ); + const body = object( + await readBoundedJson(response, undefined, interaction.signal), + "GitHub device token", + ); + if (response.ok && typeof body.access_token === "string") { + return { status: "complete", value: body.access_token }; + } + if (body.error === "authorization_pending") return { status: "pending" }; + if (body.error === "slow_down") return { status: "slow_down" }; + return { + status: "failed", + message: `GitHub device token failed with status ${response.status}`, + }; + }, + }); + return exchange(githubToken, domain, interaction.signal); + }, + refresh: (credential, signal) => + exchange(credential.refresh, credential.metadata?.domain ?? "github.com", signal), + toAuth: (credential) => ({ + apiKey: credential.access, + baseUrl: + credential.metadata?.baseUrl ?? + copilotBaseUrl(credential.access, credential.metadata?.domain ?? "github.com"), + }), + }; +} + +export function createRadiusOAuth( + gatewayUrl: string, + options: OAuthFactoryOptions = {}, +): OAuthAuthMethod { + const fetchImpl = options.fetch ?? fetch; + const now = options.now ?? Date.now; + const sleep = options.sleep ?? defaultSleep; + const gateway = new URL(gatewayUrl); + if (gateway.protocol !== "https:" && gateway.protocol !== "http:") { + throw new Error("Radius gateway must use HTTP or HTTPS"); + } + gateway.pathname = stripTrailingSlashes(gateway.pathname); + const endpoint = (path: string) => + new URL(path, `${stripTrailingSlashes(gateway.toString())}/`).toString(); + const requestToken = async ( + fields: Readonly>, + signal: AbortSignal, + operation: string, + previousRefresh?: string, + ) => + oauthCredential( + await jsonRequest( + fetchImpl, + endpoint("v1/oauth/token"), + formRequest(fields, signal), + operation, + ), + now, + operation, + previousRefresh, + ); + return { + displayName: "Radius OAuth", + login: async (interaction) => { + const method = await interaction.prompt({ + type: "select", + message: "Select Radius login method:", + options: [ + { id: "browser", label: "Browser login" }, + { id: "device_code", label: "Device code login" }, + ], + }); + if (method === "device_code") { + const start = await jsonRequest( + fetchImpl, + endpoint("v1/oauth/device"), + formRequest( + { client_id: "pi-gateway", scope: "gateway offline_access" }, + interaction.signal, + ), + "Radius device authorization", + ); + const device = deviceCode(start, "Radius device authorization"); + notifyDevice(interaction, device); + return pollFormToken({ + fetchImpl, + url: endpoint("v1/oauth/token"), + fields: { + grant_type: DEVICE_GRANT, + client_id: "pi-gateway", + device_code: device.deviceCode, + }, + device, + signal: interaction.signal, + sleep, + now, + operation: "Radius device token", + }); + } + if (method !== "browser") throw new Error(`Unknown Radius login method: ${method}`); + const discovery = await jsonRequest( + fetchImpl, + endpoint("v1/oauth"), + { headers: { accept: "application/json" }, signal: interaction.signal }, + "Radius OAuth discovery", + ); + const authorizationEndpoint = trustedUrl( + requiredString(discovery, "authorizationEndpoint", "Radius OAuth discovery"), + "Radius OAuth discovery", + ); + const { verifier, challenge } = pkce(); + const state = randomUUID(); + const redirectUri = "http://127.0.0.1:1456/oauth/callback"; + const url = new URL(authorizationEndpoint); + url.search = new URLSearchParams({ + response_type: "code", + client_id: "pi-gateway", + redirect_uri: redirectUri, + scope: "gateway offline_access", + code_challenge: challenge, + code_challenge_method: "S256", + handoff: "url", + state, + }).toString(); + const code = authorizationCode( + await promptForCode(interaction, url.toString(), redirectUri), + state, + ); + return requestToken( + { + grant_type: "authorization_code", + client_id: "pi-gateway", + redirect_uri: redirectUri, + code, + code_verifier: verifier, + }, + interaction.signal, + "Radius token exchange", + ); + }, + refresh: (credential, signal) => + requestToken( + { grant_type: "refresh_token", client_id: "pi-gateway", refresh_token: credential.refresh }, + signal, + "Radius token refresh", + credential.refresh, + ), + toAuth: (credential) => ({ apiKey: credential.access }), + }; +} + +export { PERMANENT_EXPIRY }; diff --git a/packages/ai/src/openai-chat-provider.ts b/packages/ai/src/openai-chat-provider.ts new file mode 100644 index 0000000..95aba0c --- /dev/null +++ b/packages/ai/src/openai-chat-provider.ts @@ -0,0 +1,407 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { ProviderAuthentication, ResolvedAuth } from "./auth.ts"; +import { AuthError } from "./auth.ts"; +import { safeProviderMessage } from "./diagnostics.ts"; +import type { + AuthMethod, + ModelErrorCategory, + ModelInfo, + ModelRequest, + ModelStreamEvent, +} from "./model.ts"; +import { + decodeOpenAiChatStream, + encodeOpenAiChatRequest, + OpenAiChatCodecError, +} from "./openai-chat.ts"; +import type { ModelProvider } from "./provider.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + prepareModelRequest, +} from "./request-preparation.ts"; +import { registerResolvedSecrets } from "./secret-context.ts"; +import { decodeSseStream } from "./sse.ts"; +import { raceWithSignal, safeEndpoint, safeFetch } from "./transport-safety.ts"; + +const DEFAULT_TIMEOUT_MS = 120_000; +const DEFAULT_MAX_RETRIES = 2; +const DEFAULT_MAX_RETRY_DELAY_MS = 30_000; +const MAX_RETRIES = 10; +const RETRYABLE_STATUSES = new Set([429, 500, 502, 503, 504]); +const SAFE_CONNECT_FAILURES = new Set([ + "EAI_AGAIN", + "ENOTFOUND", + "ECONNREFUSED", + "UND_ERR_CONNECT_TIMEOUT", +]); + +export interface OpenAiChatEndpoint { + url(model: ModelInfo, resolved: ResolvedAuth): string; + headers(model: ModelInfo, resolved: ResolvedAuth): Readonly>; + wireModelId?(model: ModelInfo, resolved: ResolvedAuth): string; +} + +export interface OpenAiChatProviderOptions { + readonly id: string; + readonly displayName: string; + readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; + readonly endpoint: OpenAiChatEndpoint; + readonly models: readonly ModelInfo[]; + readonly resolveAuth: (signal: AbortSignal) => Promise; + readonly fetch?: typeof fetch; + readonly now?: () => number; +} + +function nestedErrorCode(error: unknown): string | undefined { + let current = error; + for (let depth = 0; depth < 4; depth += 1) { + if (typeof current !== "object" || current === null) return undefined; + const candidate = current as { code?: unknown; cause?: unknown }; + if (typeof candidate.code === "string") return candidate.code; + current = candidate.cause; + } + return undefined; +} + +function retryAfterMs(headers: Headers, now: number): number | undefined { + const value = headers.get("retry-after")?.trim(); + if (!value) return undefined; + const seconds = Number(value); + if (Number.isFinite(seconds) && seconds >= 0) return Math.round(seconds * 1_000); + const date = Date.parse(value); + return Number.isFinite(date) ? Math.max(0, date - now) : undefined; +} + +function retryDelayMs( + attempt: number, + headers: Headers | undefined, + maxDelayMs: number, + now: number, +): number { + const advised = headers === undefined ? undefined : retryAfterMs(headers, now); + return Math.min(advised ?? 250 * 2 ** attempt, maxDelayMs); +} + +function wait(delayMs: number, signal: AbortSignal): Promise { + signal.throwIfAborted(); + if (delayMs === 0) return Promise.resolve(); + return new Promise((resolvePromise, reject) => { + const timer = setTimeout(done, delayMs); + function done(): void { + signal.removeEventListener("abort", aborted); + resolvePromise(); + } + function aborted(): void { + clearTimeout(timer); + reject(signal.reason); + } + signal.addEventListener("abort", aborted, { once: true }); + }); +} + +function statusCategory(status: number): ModelErrorCategory { + if (status === 401) return "authentication"; + if (status === 403) return "authorization"; + if (status === 408) return "timeout"; + if (status === 429) return "rate_limit"; + if (status >= 500) return "provider_internal"; + return "invalid_request"; +} + +/** Reusable HTTP and SSE transport for providers using OpenAI Chat Completions. */ +export class OpenAiChatProvider implements ModelProvider { + readonly id: string; + readonly displayName: string; + readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; + private readonly endpoint: OpenAiChatEndpoint; + private readonly models: readonly ModelInfo[]; + private readonly resolveAuth: (signal: AbortSignal) => Promise; + private readonly fetchImpl: typeof fetch | undefined; + private readonly now: () => number; + + constructor(options: OpenAiChatProviderOptions) { + this.id = options.id; + this.displayName = options.displayName; + this.authMethods = [...options.authMethods]; + if (options.authentication !== undefined) this.authentication = options.authentication; + this.endpoint = options.endpoint; + this.models = [...options.models]; + this.resolveAuth = options.resolveAuth; + this.fetchImpl = options.fetch; + this.now = options.now ?? Date.now; + } + + listModels(): Promise { + return Promise.resolve(this.models); + } + + stream(request: ModelRequest): AsyncIterable { + const model = this.models.find((candidate) => candidate.modelId === request.modelId); + if (model === undefined) { + throw new OpenAiChatCodecError(`Provider ${this.id} has no model ${request.modelId}`); + } + return this.streamModel(model, request); + } + + streamModel(model: ModelInfo, request: ModelRequest): AsyncIterable { + if ( + model.providerId !== this.id || + model.modelId !== request.modelId || + model.apiDialect !== "openai-chat" + ) { + throw new OpenAiChatCodecError(`Provider ${this.id} cannot dispatch the supplied model`); + } + return this.run(model, request); + } + + private async *run( + model: ModelInfo, + request: ModelRequest, + ): AsyncGenerator { + const timeoutSignal = AbortSignal.timeout(request.timeoutMs ?? DEFAULT_TIMEOUT_MS); + const signal = + request.signal === undefined + ? timeoutSignal + : AbortSignal.any([request.signal, timeoutSignal]); + let prepared: PreparedModelRequest; + let resolved: ResolvedAuth; + let url: string; + let init: RequestInit; + let secretValues: readonly string[] = []; + + try { + signal.throwIfAborted(); + prepared = isPreparedModelRequest(request) + ? request + : await prepareModelRequest(model, request); + resolved = await raceWithSignal(this.resolveAuth(signal), signal); + signal.throwIfAborted(); + secretValues = resolved.secretValues; + registerResolvedSecrets(secretValues); + const encoded = encodeOpenAiChatRequest( + model, + prepared, + this.endpoint.wireModelId?.(model, resolved) ?? model.modelId, + ); + url = safeEndpoint(this.endpoint.url(model, resolved), { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: this.id === "custom", + allowQuery: true, + }); + init = { + method: "POST", + headers: { + "content-type": "application/json", + accept: "text/event-stream", + ...encoded.headers, + ...this.endpoint.headers(model, resolved), + }, + body: JSON.stringify(encoded.body), + signal, + }; + } catch (error) { + yield this.failure( + request, + signal, + error, + secretValues, + "provider_request_setup_failed", + "before_dispatch", + false, + error instanceof AuthError + ? "authentication" + : error instanceof OpenAiChatCodecError + ? "invalid_request" + : "unknown", + ); + return; + } + + const maxRetries = Math.min(request.maxRetries ?? DEFAULT_MAX_RETRIES, MAX_RETRIES); + const maxRetryDelayMs = request.maxRetryDelayMs ?? DEFAULT_MAX_RETRY_DELAY_MS; + const startedAtMs = this.now(); + let response: Response | undefined; + for (let attempt = 0; attempt <= maxRetries; attempt += 1) { + try { + response = await raceWithSignal( + safeFetch(url, init, { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: this.id === "custom", + expectedOrigin: new URL(url).origin, + ...(this.fetchImpl === undefined ? {} : { fetch: this.fetchImpl }), + }), + signal, + ); + } catch (error) { + if (signal.aborted) { + yield this.failure( + request, + signal, + error, + secretValues, + "provider_request_failed", + "before_dispatch", + false, + "timeout", + ); + return; + } + const nativeCode = nestedErrorCode(error); + const retryable = nativeCode !== undefined && SAFE_CONNECT_FAILURES.has(nativeCode); + if (retryable && attempt < maxRetries) { + try { + await wait(retryDelayMs(attempt, undefined, maxRetryDelayMs, this.now()), signal); + } catch (waitError) { + yield this.failure( + request, + signal, + waitError, + secretValues, + "provider_request_failed", + "before_dispatch", + false, + "timeout", + ); + return; + } + continue; + } + yield this.failure( + request, + signal, + error, + secretValues, + "provider_request_failed", + retryable ? "before_dispatch" : "unknown", + retryable, + "network", + ); + return; + } + + if (response.ok) break; + const retryable = RETRYABLE_STATUSES.has(response.status); + const advisedDelay = retryable ? retryAfterMs(response.headers, this.now()) : undefined; + if (retryable && attempt < maxRetries) { + await response.body?.cancel(); + try { + await wait(retryDelayMs(attempt, response.headers, maxRetryDelayMs, this.now()), signal); + } catch (error) { + yield this.failure( + request, + signal, + error, + secretValues, + "provider_request_failed", + "before_dispatch", + false, + "timeout", + ); + return; + } + response = undefined; + continue; + } + await response.body?.cancel(); + yield { + type: "error", + code: `http_${response.status}`, + message: `Provider ${this.id} returned ${response.status}`, + retryable, + category: statusCategory(response.status), + requestPhase: "awaiting_response", + ...(advisedDelay === undefined ? {} : { retryAfterMs: advisedDelay }), + }; + return; + } + + if (response?.body === undefined || response.body === null) { + yield { + type: "error", + code: "empty_response", + message: `Provider ${this.id} returned no response body`, + retryable: false, + category: "provider_internal", + requestPhase: "awaiting_response", + }; + return; + } + + let emittedContent = false; + try { + const events = decodeOpenAiChatStream(decodeSseStream(response.body), { + model, + request: prepared, + startedAtMs, + now: this.now, + secretValues, + }); + const iterator = events[Symbol.asyncIterator](); + for (;;) { + const next = await raceWithSignal(iterator.next(), signal); + if (next.done) break; + const event = next.value; + if (event.type !== "completed" && event.type !== "error" && event.type !== "aborted") { + emittedContent = true; + } + yield event; + if (event.type === "completed" || event.type === "error" || event.type === "aborted") + return; + } + } catch (error) { + yield this.failure( + request, + signal, + error, + secretValues, + "provider_stream_failed", + "streaming", + false, + signal.aborted ? "timeout" : "stream_interrupted", + emittedContent, + ); + } + } + + private failure( + request: ModelRequest, + operationSignal: AbortSignal, + error: unknown, + secretValues: readonly string[], + code: string, + requestPhase: "before_dispatch" | "streaming" | "unknown", + retryable: boolean, + category: ModelErrorCategory, + partial = false, + ): ModelStreamEvent { + if (request.signal?.aborted) return { type: "aborted", ...(partial ? { partial: true } : {}) }; + if (operationSignal.aborted) { + return { + type: "error", + code: "provider_timeout", + message: `Provider ${this.id} request timed out`, + retryable: true, + category: "timeout", + requestPhase, + ...(partial ? { partial: true } : {}), + }; + } + return { + type: "error", + code, + message: safeProviderMessage( + error instanceof Error ? error.message : "provider request failed", + secretValues, + ), + retryable, + category, + requestPhase, + ...(partial ? { partial: true } : {}), + }; + } +} diff --git a/packages/ai/src/openai-chat.ts b/packages/ai/src/openai-chat.ts new file mode 100644 index 0000000..5ad6bf3 --- /dev/null +++ b/packages/ai/src/openai-chat.ts @@ -0,0 +1,954 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Axl-native OpenAI Chat Completions request and streaming response codec. + +import type { JsonObject, JsonValue, Usage } from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { ModelInfo, ModelStreamEvent, OpenAiChatCompatibility } from "./model.ts"; +import { + isPreparedModelRequest, + preparedBlobDataUrl, + type CachePlacement, + type PreparedModelRequest, + type PreparedRequestMessage, + type PreparedToolDeclaration, +} from "./request-preparation.ts"; +import type { SseFrame } from "./sse.ts"; +import { withUsageCost } from "./usage.ts"; + +const REASONING_FIELDS = new Set(["reasoning", "reasoning_content", "reasoning_text"]); +const RESERVED_REQUEST_FIELDS = new Set([ + "model", + "messages", + "stream", + "stream_options", + "store", + "tools", + "tool_choice", + "max_tokens", + "max_completion_tokens", + "reasoning_effort", + "reasoning", + "thinking", + "enable_thinking", + "chat_template_kwargs", + "chat_template_args", + "prompt_cache_key", + "prompt_cache_retention", + "temperature", + "top_p", + "top_k", + "min_p", + "frequency_penalty", + "presence_penalty", + "repetition_penalty", + "seed", + "provider", +]); + +export class OpenAiChatCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "OpenAiChatCodecError"; + } +} + +export interface EncodedOpenAiChatRequest { + readonly body: JsonObject; + /** Safe affinity headers only. Authentication remains transport-owned. */ + readonly headers: Readonly>; +} + +type MutableJsonObject = Record; + +type WireMessage = MutableJsonObject & { role: JsonValue; content?: JsonValue }; + +function isJsonObject(value: JsonValue): value is Readonly> { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function compatibility(model: ModelInfo): OpenAiChatCompatibility { + if (model.apiDialect !== "openai-chat") { + throw new OpenAiChatCodecError(`Model ${model.modelId} does not use the openai-chat dialect`); + } + if (model.compatibility?.dialect !== "openai-chat") { + throw new OpenAiChatCodecError( + `Model ${model.modelId} has no OpenAI Chat compatibility record`, + ); + } + return model.compatibility; +} + +function preparedRequest(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new OpenAiChatCodecError("OpenAI Chat requires a prepared model request"); + } +} + +function cacheControl( + model: ModelInfo, + request: PreparedModelRequest, +): MutableJsonObject | undefined { + if (request.preparation.cache.retention === "none") return undefined; + const compat = compatibility(model); + if (compat.cacheControlFormat !== "anthropic") return undefined; + if (request.preparation.cache.retention === "long") { + if (compat.supportsLongCacheRetention !== true) { + throw new OpenAiChatCodecError("Long cache retention is unsupported by this Chat endpoint"); + } + return { type: "ephemeral", ttl: "1h" }; + } + return { type: "ephemeral" }; +} + +function hasPlacement( + placements: readonly CachePlacement[], + target: CachePlacement["target"], + messageIndex?: number, + contentIndex?: number, +): boolean { + return placements.some( + (placement) => + placement.target === target && + placement.messageIndex === messageIndex && + placement.contentIndex === contentIndex, + ); +} + +function contentParts( + request: PreparedModelRequest, + message: PreparedRequestMessage, + messageIndex: number, + marker: MutableJsonObject | undefined, +): JsonValue[] { + return message.content.map((item, contentIndex): JsonValue => { + const marked = + marker !== undefined && + hasPlacement( + request.preparation.cache.placements, + "message-content", + messageIndex, + contentIndex, + ); + if (item.type === "text") { + return { type: "text", text: item.text, ...(marked ? { cache_control: marker } : {}) }; + } + if (item.type === "blob") { + const blob = request.preparation.blobs.get(item.blob.sha256); + if (blob === undefined) { + throw new OpenAiChatCodecError(`Prepared blob ${item.blob.sha256} is unavailable`); + } + return { + type: "image_url", + image_url: { url: preparedBlobDataUrl(blob) }, + ...(marked ? { cache_control: marker } : {}), + }; + } + throw new OpenAiChatCodecError("OpenAI Chat cannot encode this content block"); + }); +} + +function textFromParts(parts: readonly JsonValue[]): string { + return parts + .filter( + (part): part is { type: "text"; text: string } => + isJsonObject(part) && part.type === "text" && typeof part.text === "string", + ) + .map((part) => part.text) + .join("\n"); +} + +function reasoningDetails(signature: string): JsonValue[] | undefined { + try { + const value = JSON.parse(signature) as unknown; + if (!Array.isArray(value) || value.length === 0) return undefined; + for (const detail of value) { + if (typeof detail !== "object" || detail === null || Array.isArray(detail)) return undefined; + const type = (detail as Record).type; + if ( + type !== "reasoning.summary" && + type !== "reasoning.encrypted" && + type !== "reasoning.text" + ) { + return undefined; + } + } + return value as JsonValue[]; + } catch { + return undefined; + } +} + +function rejectContinuation(message: PreparedRequestMessage, messageIndex: number): void { + if (message.role !== "assistant") return; + if (message.continuation !== undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}] has continuation metadata unsupported by OpenAI Chat`, + ); + } + for (const [contentIndex, content] of message.content.entries()) { + if (content.type === "text" && content.continuation !== undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}].content[${contentIndex}] has unsupported continuation metadata`, + ); + } + } + for (const [callIndex, call] of (message.toolCalls ?? []).entries()) { + if (call.continuation !== undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}].toolCalls[${callIndex}] has unsupported continuation metadata`, + ); + } + } +} + +function assistantMessage( + request: PreparedModelRequest, + message: Extract, + messageIndex: number, + compat: OpenAiChatCompatibility, + marker: MutableJsonObject | undefined, +): WireMessage { + rejectContinuation(message, messageIndex); + if (message.content.some((part) => part.type === "blob")) { + throw new OpenAiChatCodecError("OpenAI Chat cannot replay image content from an assistant"); + } + const textParts: JsonValue[] = message.content.flatMap((part, contentIndex) => + part.type === "text" + ? [ + { + type: "text", + text: part.text, + ...(marker !== undefined && + hasPlacement( + request.preparation.cache.placements, + "message-content", + messageIndex, + contentIndex, + ) + ? { cache_control: marker } + : {}), + }, + ] + : [], + ); + const text = textFromParts(textParts); + const hasMarkedText = textParts.some( + (part) => isJsonObject(part) && part.cache_control !== undefined, + ); + const thinking = message.content.filter((part) => part.type === "thinking"); + const wire: WireMessage = { + role: "assistant", + content: text.length > 0 ? (hasMarkedText ? textParts : text) : null, + }; + + if (thinking.length > 0) { + const signedDetails: JsonValue[] = []; + for (const part of thinking) { + if (part.signature === undefined || REASONING_FIELDS.has(part.signature.value)) continue; + const parsed = reasoningDetails(part.signature.value); + if (parsed === undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}] has an unrecognized OpenAI Chat reasoning signature`, + ); + } + signedDetails.push(...parsed); + } + if (signedDetails.length > 0) { + wire.reasoning_details = signedDetails; + } else if (compat.requiresThinkingAsText === true) { + const thinkingText = thinking + .map((part) => part.text) + .filter(Boolean) + .join("\n\n"); + wire.content = [ + ...(thinkingText.length === 0 ? [] : [{ type: "text", text: thinkingText }]), + ...textParts, + ]; + } else { + for (const part of thinking) { + if (part.signature !== undefined && !REASONING_FIELDS.has(part.signature.value)) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}] has an unrecognized OpenAI Chat reasoning signature`, + ); + } + } + const field = thinking.find((part) => part.signature !== undefined)?.signature?.value; + const reasoningField = field ?? "reasoning_content"; + wire[reasoningField] = thinking.map((part) => part.text).join("\n"); + } + } + + if (message.toolCalls !== undefined && message.toolCalls.length > 0) { + const grammarTools = new Map( + request.preparation.tools + .filter((tool) => tool.preparedConstraint?.type === "grammar") + .map((tool) => [tool.name, tool]), + ); + wire.tool_calls = message.toolCalls.map((call): JsonValue => { + if (call.signature !== undefined) { + const detail = reasoningDetails(call.signature.value); + if (detail === undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}] has an unrecognized tool reasoning signature`, + ); + } + const existing = wire.reasoning_details; + wire.reasoning_details = [...(Array.isArray(existing) ? existing : []), ...detail]; + } + const grammar = grammarTools.get(call.name)?.preparedConstraint; + if (grammar?.type === "grammar") { + const value = call.input[grammar.inputProperty]; + if (typeof value !== "string") { + throw new OpenAiChatCodecError( + `Grammar tool ${call.canonicalName} needs string input ${grammar.inputProperty}`, + ); + } + return { id: call.callId, type: "custom", custom: { name: call.name, input: value } }; + } + return { + id: call.callId, + type: "function", + function: { name: call.name, arguments: JSON.stringify(call.input) }, + }; + }); + } + + if ( + compat.requiresReasoningContentOnAssistantMessages === true && + wire.reasoning_content === undefined + ) { + wire.reasoning_content = ""; + } + const hasContent = + typeof wire.content === "string" + ? wire.content.length > 0 + : Array.isArray(wire.content) && wire.content.length > 0; + if (!hasContent && wire.tool_calls === undefined && wire.reasoning_details === undefined) { + throw new OpenAiChatCodecError( + `messages[${messageIndex}] has no OpenAI Chat renderable assistant content`, + ); + } + return wire; +} + +function instructionMessage( + model: ModelInfo, + request: PreparedModelRequest, + marker: MutableJsonObject | undefined, +): WireMessage | undefined { + if (request.system === undefined || request.system.length === 0) return undefined; + const compat = compatibility(model); + const role = model.reasoning && compat.supportsDeveloperRole === true ? "developer" : "system"; + const marked = + marker !== undefined && hasPlacement(request.preparation.cache.placements, "system"); + return { + role, + content: marked + ? [{ type: "text", text: request.system, cache_control: marker }] + : request.system, + }; +} + +function encodeMessages( + model: ModelInfo, + request: PreparedModelRequest, + marker: MutableJsonObject | undefined, +): JsonValue[] { + const compat = compatibility(model); + const output: JsonValue[] = []; + const pendingToolImages: JsonValue[] = []; + const instruction = instructionMessage(model, request, marker); + if (instruction !== undefined) output.push(instruction); + + for (let messageIndex = 0; messageIndex < request.messages.length; messageIndex += 1) { + const message = request.messages[messageIndex]; + if (message === undefined) continue; + if (message.role === "user") { + const parts = contentParts(request, message, messageIndex, marker); + const onlyPart = parts[0]; + const plainText = + parts.length === 1 && + onlyPart !== undefined && + isJsonObject(onlyPart) && + onlyPart.type === "text" && + onlyPart.cache_control === undefined; + output.push({ role: "user", content: plainText ? (onlyPart.text ?? "") : parts }); + continue; + } + if (message.role === "assistant") { + output.push(assistantMessage(request, message, messageIndex, compat, marker)); + continue; + } + + const imageParts: JsonValue[] = []; + const parts = contentParts(request, message, messageIndex, marker); + for (const part of parts) { + if (isJsonObject(part) && part.type === "image_url") imageParts.push(part); + } + const text = textFromParts(parts); + const markedText = parts.some( + (part) => isJsonObject(part) && part.type === "text" && part.cache_control !== undefined, + ); + const toolContent = + text.length > 0 + ? markedText + ? parts.filter((part) => isJsonObject(part) && part.type === "text") + : text + : imageParts.length > 0 + ? "(see attached image)" + : "(no tool output)"; + const toolMessage: MutableJsonObject = { + role: "tool", + tool_call_id: message.callId, + content: toolContent, + ...(compat.requiresToolResultName === true ? { name: message.name } : {}), + }; + output.push(toolMessage); + pendingToolImages.push(...imageParts); + const nextMessage = request.messages[messageIndex + 1]; + if (nextMessage?.role !== "tool" && pendingToolImages.length > 0) { + output.push({ + role: "user", + content: [ + { type: "text", text: "Attached image(s) from tool result:" }, + ...pendingToolImages, + ], + }); + pendingToolImages.length = 0; + } else if (nextMessage?.role === "user" && compat.requiresAssistantAfterToolResult === true) { + output.push({ role: "assistant", content: "I have processed the tool results." }); + } + } + return output; +} + +function encodeTools( + request: PreparedModelRequest, + marker: MutableJsonObject | undefined, +): JsonValue[] | undefined { + if (request.tools === undefined || request.tools.length === 0) return undefined; + return request.tools.map((tool, toolIndex): JsonValue => { + let rendered: MutableJsonObject; + if (tool.preparedConstraint?.type === "grammar") { + rendered = { + type: "custom", + custom: { + name: tool.name, + description: tool.description, + format: { + type: "grammar", + grammar: { + syntax: tool.preparedConstraint.format, + definition: tool.preparedConstraint.definition, + }, + }, + }, + }; + } else { + rendered = { + type: "function", + function: { + name: tool.name, + description: tool.description, + parameters: tool.inputSchema, + ...(tool.preparedConstraint?.type === "json-schema" + ? { strict: tool.preparedConstraint.strict } + : {}), + }, + }; + } + const marked = request.preparation.cache.placements.some( + (placement) => placement.target === "tool" && placement.toolIndex === toolIndex, + ); + if (marked && marker !== undefined) rendered.cache_control = marker; + return rendered; + }); +} + +function applyReasoningControl( + body: MutableJsonObject, + model: ModelInfo, + request: PreparedModelRequest, +): void { + const reasoning = request.preparation.reasoning; + if (reasoning === undefined) return; + const compat = compatibility(model); + const enabled = reasoning.effective !== "off"; + const effort = reasoning.providerValue; + + if (compat.thinkingFormat === "openrouter") { + body.reasoning = { effort: enabled ? (effort ?? reasoning.effective) : (effort ?? "none") }; + } else if (compat.thinkingFormat === "deepseek") { + body.thinking = { type: enabled ? "enabled" : "disabled" }; + if (enabled && compat.supportsReasoningEffort === true) { + body.reasoning_effort = effort ?? reasoning.effective; + } + } else if (compat.thinkingFormat === "together") { + body.reasoning = { enabled }; + if (enabled && compat.supportsReasoningEffort === true) { + body.reasoning_effort = effort ?? reasoning.effective; + } + } else if (compat.thinkingFormat === "zai") { + body.thinking = enabled ? { type: "enabled", clear_thinking: false } : { type: "disabled" }; + if (enabled && compat.supportsReasoningEffort === true) { + body.reasoning_effort = effort ?? reasoning.effective; + } + } else if (compat.thinkingFormat === "qwen") { + body.enable_thinking = enabled; + if (enabled && compat.supportsReasoningEffort === true) { + body.reasoning_effort = effort ?? reasoning.effective; + } + } else if (compat.thinkingFormat === "chat-template") { + body.chat_template_kwargs = { enable_thinking: enabled }; + } else if (compat.thinkingFormat === "baseten") { + body.chat_template_args = { enable_thinking: enabled }; + if (compat.supportsReasoningEffort === true && (enabled || effort !== undefined)) { + body.reasoning_effort = effort ?? reasoning.effective; + } + } else if (compat.thinkingFormat === "string-thinking") { + body.thinking = effort ?? (enabled ? reasoning.effective : "none"); + } else if (compat.thinkingFormat === "ant-ling") { + if (enabled) { + if (effort === undefined) { + throw new OpenAiChatCodecError("Ant Ling reasoning requires a prepared provider effort"); + } + body.reasoning = { effort }; + } + } else if (compat.supportsReasoningEffort === true) { + if (enabled) body.reasoning_effort = effort ?? reasoning.effective; + else if (effort !== undefined) body.reasoning_effort = effort; + } else if (enabled) { + throw new OpenAiChatCodecError("This Chat endpoint cannot render prepared reasoning controls"); + } + + if (reasoning.tokenBudget !== undefined) { + const field = compat.thinkingTokenBudgetField; + if (field === undefined) { + throw new OpenAiChatCodecError("Prepared reasoning budget has no Chat request field"); + } + body[field] = reasoning.tokenBudget; + } +} + +function applySampling(body: MutableJsonObject, request: PreparedModelRequest): void { + const sampling = request.sampling; + if (sampling === undefined) return; + const fields = { + temperature: "temperature", + topP: "top_p", + topK: "top_k", + minP: "min_p", + frequencyPenalty: "frequency_penalty", + presencePenalty: "presence_penalty", + repetitionPenalty: "repetition_penalty", + seed: "seed", + } as const; + for (const [source, target] of Object.entries(fields) as [keyof typeof fields, string][]) { + const value = sampling[source]; + if (value !== undefined) body[target] = value; + } + for (const [field, value] of Object.entries(sampling.custom ?? {})) { + if (RESERVED_REQUEST_FIELDS.has(field) || field in body) { + throw new OpenAiChatCodecError( + `Custom sampling field ${field} collides with a request field`, + ); + } + body[field] = value; + } +} + +function applyCache( + body: MutableJsonObject, + headers: Record, + model: ModelInfo, + request: PreparedModelRequest, +): void { + const cache = request.preparation.cache; + if (cache.retention === "none") return; + const compat = compatibility(model); + if (cache.retention === "long") { + if (compat.supportsLongCacheRetention !== true) { + throw new OpenAiChatCodecError("Long cache retention is unsupported by this Chat endpoint"); + } + body.prompt_cache_retention = "24h"; + } + if (cache.sessionId === undefined) return; + if (model.providerId === "openai" || compat.supportsLongCacheRetention === true) { + body.prompt_cache_key = Array.from(cache.sessionId).slice(0, 64).join(""); + } + if (compat.sessionAffinityFormat === "openrouter") { + headers["x-session-id"] = cache.sessionId; + } else if (compat.sessionAffinityFormat === "openai") { + headers.session_id = cache.sessionId; + headers["x-client-request-id"] = cache.sessionId; + headers["x-session-affinity"] = cache.sessionId; + } else if (compat.sessionAffinityFormat === "openai-no-session") { + headers["x-client-request-id"] = cache.sessionId; + headers["x-session-affinity"] = cache.sessionId; + } +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeOpenAiChatRequest( + model: ModelInfo, + request: PreparedModelRequest, + wireModelId = model.modelId, +): EncodedOpenAiChatRequest { + preparedRequest(request); + const compat = compatibility(model); + if (request.modelId !== model.modelId) { + throw new OpenAiChatCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (request.metadata !== undefined && Object.keys(request.metadata).length > 0) { + throw new OpenAiChatCodecError("OpenAI Chat cannot render request metadata"); + } + + const marker = cacheControl(model, request); + const body: MutableJsonObject = { + model: wireModelId, + messages: encodeMessages(model, request, marker), + stream: true, + }; + if (compat.supportsUsageInStreaming !== false) body.stream_options = { include_usage: true }; + if (compat.supportsStore === true) body.store = false; + if (request.maxOutputTokens !== undefined) { + body[compat.maxTokensField ?? "max_completion_tokens"] = request.maxOutputTokens; + } + const tools = encodeTools(request, marker); + if (tools !== undefined) body.tools = tools; + if (request.toolChoice !== undefined) { + if (request.toolChoice !== "none" && tools === undefined) { + throw new OpenAiChatCodecError(`toolChoice ${request.toolChoice} needs at least one tool`); + } + body.tool_choice = request.toolChoice; + } + applyReasoningControl(body, model, request); + applySampling(body, request); + const headers: Record = {}; + applyCache(body, headers, model, request); + return { body, headers }; +} + +interface ToolAccumulator { + readonly index: number; + readonly contentIndex: number; + id: string; + name: string; + arguments: string; + customInput: string; + custom: boolean; + started: boolean; + emittedArguments: number; +} + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function usage(raw: unknown, model: ModelInfo): Usage { + const value = object(raw) ?? {}; + const promptDetails = object(value.prompt_tokens_details); + const completionDetails = object(value.completion_tokens_details); + const prompt = typeof value.prompt_tokens === "number" ? value.prompt_tokens : 0; + const cacheRead = + typeof promptDetails?.cached_tokens === "number" + ? promptDetails.cached_tokens + : typeof value.prompt_cache_hit_tokens === "number" + ? value.prompt_cache_hit_tokens + : typeof value.cached_tokens === "number" + ? value.cached_tokens + : 0; + const cacheWrite = + typeof promptDetails?.cache_write_tokens === "number" ? promptDetails.cache_write_tokens : 0; + const output = typeof value.completion_tokens === "number" ? value.completion_tokens : 0; + const reasoning = + typeof completionDetails?.reasoning_tokens === "number" + ? completionDetails.reasoning_tokens + : 0; + const mapped: Usage = { + inputTokens: Math.max(0, prompt - cacheRead - cacheWrite), + outputTokens: output, + cacheReadTokens: cacheRead, + cacheWriteTokens: cacheWrite, + reasoningTokens: reasoning, + }; + return model.cost === undefined ? mapped : withUsageCost(model.cost, mapped); +} + +function reverseToolName(request: PreparedModelRequest, name: string): string { + return request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? name; +} + +function grammarTool( + request: PreparedModelRequest, + name: string, +): PreparedToolDeclaration | undefined { + return request.preparation.tools.find( + (tool) => tool.name === name && tool.preparedConstraint?.type === "grammar", + ); +} + +function retryableProviderCode(code: string): boolean { + return code === "rate_limit_exceeded" || code === "server_error" || code === "timeout"; +} + +export interface OpenAiChatDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; +} + +/** Decodes Chat Completions SSE frames into canonical stream events. */ +export async function* decodeOpenAiChatStream( + frames: AsyncIterable, + options: OpenAiChatDecodeOptions, +): AsyncGenerator { + preparedRequest(options.request); + compatibility(options.model); + const tools = new Map(); + let finalUsage = usage(undefined, options.model); + let finishReason: string | undefined; + let responseId: string | undefined; + let routedModelId: string | undefined; + let emittedContent = false; + let sawToolCall = false; + let nextContentIndex = 0; + let textContentIndex: number | undefined; + let thinkingContentIndex: number | undefined; + + const responseMetadata = () => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined ? {} : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + ...(finishReason === undefined ? {} : { nativeStopReason: finishReason }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }); + + const terminal = (): ModelStreamEvent => { + if (finishReason === "stop" || finishReason === "end") { + return { + type: "completed", + stopReason: "stop", + usage: finalUsage, + response: responseMetadata(), + }; + } + if (finishReason === "length") { + return { + type: "completed", + stopReason: "length", + usage: finalUsage, + partial: true, + response: responseMetadata(), + }; + } + if (finishReason === "tool_calls" || finishReason === "function_call") { + return { + type: "completed", + stopReason: "tool_use", + usage: finalUsage, + response: responseMetadata(), + }; + } + return { + type: "error", + code: "provider_finish_reason", + message: `Provider finish reason: ${finishReason ?? "missing"}`, + retryable: finishReason === "network_error", + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + }; + + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (frame.data === "[DONE]") break; + let chunk: Record; + try { + const parsed = JSON.parse(frame.data) as unknown; + const parsedObject = object(parsed); + if (parsedObject === undefined) throw new Error("frame is not an object"); + chunk = parsedObject; + } catch (error) { + throw new OpenAiChatCodecError("Provider sent an undecodable Chat stream frame", { + cause: error, + }); + } + + const providerError = object(chunk.error); + if (providerError !== undefined) { + const code = String(providerError.code ?? providerError.type ?? "provider_error"); + const message = safeProviderMessage( + typeof providerError.message === "string" + ? providerError.message + : "Provider reported a failure", + options.secretValues, + ); + yield { + type: "error", + code, + message, + retryable: retryableProviderCode(code), + ...(emittedContent ? { partial: true } : {}), + response: responseMetadata(), + }; + return; + } + + if (typeof chunk.id === "string" && chunk.id.length > 0) responseId ??= chunk.id; + if (typeof chunk.model === "string" && chunk.model.length > 0) routedModelId ??= chunk.model; + if (chunk.usage !== undefined) finalUsage = usage(chunk.usage, options.model); + + if (chunk.choices === undefined) continue; + if (!Array.isArray(chunk.choices)) { + throw new OpenAiChatCodecError("Provider Chat stream choices must be an array"); + } + if (chunk.choices.length === 0) continue; + if (chunk.choices.length !== 1) { + throw new OpenAiChatCodecError("Provider returned multiple Chat completion choices"); + } + const choice = object(chunk.choices[0]); + if (choice === undefined) + throw new OpenAiChatCodecError("Provider returned a malformed Chat choice"); + if (choice.usage !== undefined && chunk.usage === undefined) { + finalUsage = usage(choice.usage, options.model); + } + if (typeof choice.finish_reason === "string" && choice.finish_reason.length > 0) { + finishReason = choice.finish_reason; + } + const delta = object(choice.delta); + if (delta === undefined) continue; + + if (typeof delta.content === "string" && delta.content.length > 0) { + textContentIndex ??= nextContentIndex++; + emittedContent = true; + yield { type: "text_delta", text: delta.content, contentIndex: textContentIndex }; + } + if (delta.reasoning_details !== undefined) { + throw new OpenAiChatCodecError( + "Provider returned reasoning replay metadata that the canonical stream cannot retain", + ); + } + for (const field of ["reasoning_content", "reasoning", "reasoning_text"]) { + const value = delta[field]; + if (typeof value === "string" && value.length > 0) { + thinkingContentIndex ??= nextContentIndex++; + emittedContent = true; + yield { type: "thinking_delta", text: value, contentIndex: thinkingContentIndex }; + break; + } + } + + if (delta.tool_calls === undefined) continue; + if (!Array.isArray(delta.tool_calls)) { + throw new OpenAiChatCodecError("Provider Chat tool call deltas must be an array"); + } + for (const rawTool of delta.tool_calls) { + const tool = object(rawTool); + if (tool === undefined || !Number.isSafeInteger(tool.index) || (tool.index as number) < 0) { + throw new OpenAiChatCodecError("Provider sent a tool call without a valid index"); + } + const index = tool.index as number; + let state = tools.get(index); + if (state === undefined) { + state = { + index, + contentIndex: nextContentIndex++, + id: "", + name: "", + arguments: "", + customInput: "", + custom: false, + started: false, + emittedArguments: 0, + }; + tools.set(index, state); + } + if (typeof tool.id === "string" && tool.id.length > 0) state.id ||= tool.id; + const functionCall = object(tool.function); + const customCall = object(tool.custom); + if (typeof functionCall?.name === "string") state.name ||= functionCall.name; + if (typeof functionCall?.arguments === "string") state.arguments += functionCall.arguments; + if (typeof customCall?.name === "string") state.name ||= customCall.name; + if (typeof customCall?.input === "string") { + state.custom = true; + state.customInput += customCall.input; + } + if (!state.started && state.id.length > 0 && state.name.length > 0) { + state.started = true; + emittedContent = true; + yield { + type: "tool_call_start", + contentIndex: state.contentIndex, + callId: state.id, + name: reverseToolName(options.request, state.name), + }; + } + const argumentsText = state.custom ? state.customInput : state.arguments; + if (state.started && argumentsText.length > state.emittedArguments) { + const argumentsDelta = argumentsText.slice(state.emittedArguments); + state.emittedArguments = argumentsText.length; + yield { + type: "tool_call_delta", + contentIndex: state.contentIndex, + callId: state.id, + argumentsDelta, + }; + } + } + } + + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } + if (finishReason === undefined) return; + for (const state of [...tools.values()].sort((left, right) => left.index - right.index)) { + if (!state.started) throw new OpenAiChatCodecError("Provider ended an incomplete tool call"); + let input: JsonObject; + if (state.custom) { + const declaration = grammarTool(options.request, state.name); + const constraint = declaration?.preparedConstraint; + if (constraint?.type !== "grammar") { + throw new OpenAiChatCodecError(`Provider returned unknown custom tool ${state.name}`); + } + input = { [constraint.inputProperty]: state.customInput }; + } else { + try { + const parsed = state.arguments.length === 0 ? {} : (JSON.parse(state.arguments) as unknown); + const parsedObject = object(parsed); + if (parsedObject === undefined) throw new Error("arguments are not an object"); + input = parsedObject as JsonObject; + } catch (error) { + throw new OpenAiChatCodecError(`Tool call ${state.id} has undecodable arguments`, { + cause: error, + }); + } + } + sawToolCall = true; + yield { + type: "tool_call", + contentIndex: state.contentIndex, + callId: state.id, + name: reverseToolName(options.request, state.name), + input, + }; + } + if (sawToolCall && finishReason === "stop") finishReason = "tool_calls"; + yield terminal(); +} diff --git a/packages/ai/src/openai-codex-responses.ts b/packages/ai/src/openai-codex-responses.ts new file mode 100644 index 0000000..33bb60d --- /dev/null +++ b/packages/ai/src/openai-codex-responses.ts @@ -0,0 +1,225 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// OpenAI Codex subscription policy around the shared Responses codec. + +import type { JsonObject, JsonValue } from "@axl/protocol"; + +import type { ResolvedAuth } from "./auth.ts"; +import type { ModelInfo } from "./model.ts"; +import { + decodeResponsesStream, + encodeResponsesRequest, + ResponsesCodecError, + type ResponsesDecodeOptions, +} from "./openai-responses.ts"; +import { isPreparedModelRequest, type PreparedModelRequest } from "./request-preparation.ts"; +import { stripTrailingSlashes } from "./transport-safety.ts"; +import type { SseFrame } from "./sse.ts"; + +const DEFAULT_CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"; +const CODEX_ACCOUNT_CLAIM = "https://api.openai.com/auth"; +const REQUIRED_CODEX_HEADERS = new Set([ + "authorization", + "chatgpt-account-id", + "originator", + "user-agent", + "openai-beta", + "accept", + "content-type", + "session-id", + "x-client-request-id", +]); + +export class OpenAiCodexResponsesCodecError extends ResponsesCodecError { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "OpenAiCodexResponsesCodecError"; + } +} + +export interface EncodedOpenAiCodexResponsesRequest { + readonly url: string; + readonly headers: Readonly>; + readonly body: JsonObject; +} + +function record(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function requireCodex(model: ModelInfo, request: PreparedModelRequest): void { + if (model.apiDialect !== "openai-codex-responses") { + throw new OpenAiCodexResponsesCodecError( + `Model ${model.modelId} does not use the OpenAI Codex Responses dialect`, + ); + } + if (!isPreparedModelRequest(request)) { + throw new OpenAiCodexResponsesCodecError( + "OpenAI Codex Responses requires a prepared model request", + ); + } +} + +function decodeBase64Url(value: string): string { + const normalized = value.replace(/-/g, "+").replace(/_/g, "/"); + const padding = "=".repeat((4 - (normalized.length % 4)) % 4); + return Buffer.from(`${normalized}${padding}`, "base64").toString("utf8"); +} + +/** Extracts the subscription account identity from an OpenAI access token. */ +export function extractOpenAiCodexAccountId(token: string): string { + try { + const segments = token.split("."); + if (segments.length !== 3 || segments[1] === undefined) throw new Error("invalid JWT"); + const payload = record(JSON.parse(decodeBase64Url(segments[1])) as unknown); + const auth = record(payload?.[CODEX_ACCOUNT_CLAIM]); + const accountId = auth?.chatgpt_account_id; + if (typeof accountId !== "string" || accountId.trim().length === 0) { + throw new Error("missing account claim"); + } + return accountId; + } catch (cause) { + throw new OpenAiCodexResponsesCodecError( + "OpenAI Codex access token has no valid ChatGPT account identity", + { cause }, + ); + } +} + +/** Resolves the Codex Responses endpoint without guessing proxy path rewrites. */ +export function openAiCodexResponsesUrl(baseUrl?: string): string { + const base = stripTrailingSlashes(baseUrl?.trim() || DEFAULT_CODEX_BASE_URL); + let url: URL; + try { + url = new URL(base); + } catch (cause) { + throw new OpenAiCodexResponsesCodecError(`Invalid OpenAI Codex base URL: ${baseUrl}`, { + cause, + }); + } + if (url.pathname.endsWith("/codex/responses")) return url.toString(); + if (url.pathname.endsWith("/codex")) url.pathname = `${url.pathname}/responses`; + else url.pathname = `${url.pathname}/codex/responses`; + return url.toString(); +} + +function modelBaseUrl(model: ModelInfo): string | undefined { + return model.endpoint?.type === "fixed" ? model.endpoint.baseUrl : undefined; +} + +function composeHeaders( + model: ModelInfo, + request: PreparedModelRequest, + resolved: ResolvedAuth, +): Readonly> { + const token = resolved.auth.apiKey; + if (token === undefined || token.length === 0) { + throw new OpenAiCodexResponsesCodecError("OpenAI Codex requires a resolved subscription token"); + } + const headers: Record = {}; + for (const source of [model.headers, resolved.auth.headers]) { + for (const [name, value] of Object.entries(source ?? {})) { + const normalized = name.toLowerCase(); + if (!REQUIRED_CODEX_HEADERS.has(normalized)) headers[normalized] = value; + } + } + headers.authorization = `Bearer ${token}`; + headers["chatgpt-account-id"] = extractOpenAiCodexAccountId(token); + headers.originator = "axl"; + headers["user-agent"] = "axl"; + headers["openai-beta"] = "responses=experimental"; + headers.accept = "text/event-stream"; + headers["content-type"] = "application/json"; + + const sessionId = request.preparation.cache.sessionId; + if (sessionId !== undefined) { + const clamped = Array.from(sessionId).slice(0, 64).join(""); + headers["session-id"] = clamped; + headers["x-client-request-id"] = clamped; + } + return headers; +} + +function codexTools(tools: JsonValue | undefined): JsonValue | undefined { + if (!Array.isArray(tools)) return tools; + return tools.map((tool): JsonValue => { + const value = record(tool); + if (value?.type !== "function" || value.strict === true) return tool; + return { ...(tool as JsonObject), strict: null }; + }); +} + +/** Composes a stateless Codex request from one immutable prepared request. */ +export function encodeOpenAiCodexResponsesRequest( + model: ModelInfo, + request: PreparedModelRequest, + resolved: ResolvedAuth, +): EncodedOpenAiCodexResponsesRequest { + requireCodex(model, request); + const shared = encodeResponsesRequest(model, request); + const body: Record = { + ...shared.body, + instructions: request.system || "You are a helpful assistant.", + include: ["reasoning.encrypted_content"], + text: { verbosity: "low" }, + tool_choice: request.toolChoice ?? "auto", + parallel_tool_calls: true, + }; + const tools = codexTools(body.tools); + if (tools !== undefined) body.tools = tools; + + return { + url: openAiCodexResponsesUrl(resolved.auth.baseUrl ?? modelBaseUrl(model)), + headers: composeHeaders(model, request, resolved), + body, + }; +} + +async function* mapCodexFrames(frames: AsyncIterable): AsyncGenerator { + for await (const frame of frames) { + if (frame.data === "[DONE]") { + yield frame; + continue; + } + let event: Record | undefined; + try { + event = record(JSON.parse(frame.data) as unknown); + } catch { + yield frame; + continue; + } + if (event?.type !== "response.done") { + yield frame; + continue; + } + const response = record(event.response); + const status = response?.status; + if (status === "completed") { + yield { ...frame, data: JSON.stringify({ ...event, type: "response.completed" }) }; + return; + } + if (status === "incomplete") { + yield { ...frame, data: JSON.stringify({ ...event, type: "response.incomplete" }) }; + return; + } + if (status === "failed" || status === "cancelled") { + yield { ...frame, data: JSON.stringify({ ...event, type: "response.failed" }) }; + return; + } + throw new OpenAiCodexResponsesCodecError( + "OpenAI Codex response.done has no supported terminal status", + ); + } +} + +/** Decodes Codex event aliases through the canonical shared Responses decoder. */ +export async function* decodeOpenAiCodexResponsesStream( + frames: AsyncIterable, + options: ResponsesDecodeOptions, +): AsyncGenerator { + requireCodex(options.model, options.request); + yield* decodeResponsesStream(mapCodexFrames(frames), options); +} diff --git a/packages/ai/src/openai-responses.ts b/packages/ai/src/openai-responses.ts index 6630a79..fc82d6c 100644 --- a/packages/ai/src/openai-responses.ts +++ b/packages/ai/src/openai-responses.ts @@ -1,43 +1,37 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan // SPDX-FileCopyrightText: 2026 Kaushik Kumar -// SPDX-FileCopyrightText: 2026 Shaan Narendran // SPDX-License-Identifier: Apache-2.0 -// Axl-native OpenAI Responses codec and transport implementation. +// Axl-native OpenAI Responses codec and legacy transport composition. -import type { - BlobReference, - JsonObject, - JsonValue, - ModelErrorCategory, - Usage, -} from "@axl/protocol"; - -import { EnvHttpProxyAgent, fetch as modelFetch } from "undici"; -import { fitModelRequest } from "./request-configuration.ts"; +import type { JsonObject, JsonValue, ModelErrorCategory, Usage } from "@axl/protocol"; -import { AuthError, type ResolvedAuth } from "./auth.ts"; +import { AuthError, type ProviderAuthentication, type ResolvedAuth } from "./auth.ts"; import { assertModelSupports } from "./capabilities.ts"; -import type { AuthMethod, ModelInfo, ModelRequest, ModelStreamEvent } from "./model.ts"; +import { safeProviderMessage } from "./diagnostics.ts"; +import type { + AuthMethod, + ModelInfo, + ModelRequest, + ModelStreamEvent, + OpenAiResponsesCompatibility, +} from "./model.ts"; import type { ModelProvider } from "./provider.ts"; +import { + isPreparedModelRequest, + type PreparedModelRequest, + type PreparedRequestMessage, + type PreparedToolDeclaration, + preparedBlobDataUrl, + prepareModelRequest, +} from "./request-preparation.ts"; +import { registerResolvedSecrets } from "./secret-context.ts"; import { decodeSseStream, type SseFrame } from "./sse.ts"; +import { safeEndpoint, safeFetch } from "./transport-safety.ts"; +import { withUsageCost } from "./usage.ts"; /** OpenAI Responses rejects max_output_tokens below 16. */ const MIN_OUTPUT_TOKENS = 16; -// Independently implements Pi's byte-idle timeout semantics from http-dispatcher.ts at 6c87d9a02. -// https://github.com/badlogic/pi-mono/blob/6c87d9a02/packages/coding-agent/src/core/http-dispatcher.ts -// Model-only connection pooling. Per-dispatch overrides also override fetch's internal defaults. -let modelDispatcher: EnvHttpProxyAgent | undefined; -function dispatcherFor(timeoutMs: number) { - modelDispatcher ??= new EnvHttpProxyAgent({ - allowH2: false, - connect: { autoSelectFamilyAttemptTimeout: 2_000 }, - }); - return modelDispatcher.compose( - (dispatch) => (options, handler) => - dispatch({ ...options, headersTimeout: timeoutMs, bodyTimeout: timeoutMs }, handler), - ); -} const IDLE_TIMEOUT_CODES = new Set(["UND_ERR_HEADERS_TIMEOUT", "UND_ERR_BODY_TIMEOUT"]); const SAFE_CONNECT_FAILURES = new Set([ "EAI_AGAIN", @@ -52,295 +46,739 @@ const RATE_LIMIT_CODES = new Set([ "too_many_requests", ]); const OVERLOADED_CODES = new Set(["overloaded", "server_error", "temporarily_unavailable"]); +const RESERVED_REQUEST_FIELDS = new Set([ + "model", + "input", + "instructions", + "stream", + "store", + "include", + "tools", + "tool_choice", + "max_output_tokens", + "reasoning", + "temperature", + "top_p", + "prompt_cache_key", + "prompt_cache_retention", +]); -function retryAfterMs(headers: Headers, now = Date.now()): number | undefined { - const value = headers.get("retry-after")?.trim(); - if (!value) return undefined; - const seconds = Number(value); - if (Number.isFinite(seconds) && seconds >= 0) return Math.round(seconds * 1_000); - const date = Date.parse(value); - return Number.isFinite(date) ? Math.max(0, date - now) : undefined; +export class ResponsesCodecError extends Error { + constructor(message: string, options?: { cause?: unknown }) { + super(message, options); + this.name = "ResponsesCodecError"; + } } -function nestedErrorCode(error: unknown): string | undefined { - let current = error; - for (let depth = 0; depth < 4; depth += 1) { - if (typeof current !== "object" || current === null) return undefined; - const candidate = current as { code?: unknown; cause?: unknown }; - if (typeof candidate.code === "string") return candidate.code; - current = candidate.cause; +export interface EncodedResponsesRequest { + readonly body: JsonObject; + /** Safe affinity headers only. Authentication remains transport-owned. */ + readonly headers: Readonly>; +} + +type MutableJsonObject = Record; + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function compatibility(model: ModelInfo): OpenAiResponsesCompatibility { + if ( + model.apiDialect !== "openai-responses" && + model.apiDialect !== "azure-openai-responses" && + model.apiDialect !== "openai-codex-responses" + ) { + throw new ResponsesCodecError(`Model ${model.modelId} does not use a Responses API dialect`); } - return undefined; + const declared = model.compatibility; + if (declared === undefined) { + return { + dialect: + model.apiDialect === "openai-responses" + ? "openai-responses" + : model.apiDialect === "azure-openai-responses" + ? "azure-openai-responses" + : "openai-codex-responses", + }; + } + if ( + (declared.dialect !== "openai-responses" && + declared.dialect !== "azure-openai-responses" && + declared.dialect !== "openai-codex-responses") || + declared.dialect !== model.apiDialect + ) { + throw new ResponsesCodecError( + `Model ${model.modelId} has no matching Responses compatibility record`, + ); + } + return declared; } -function providerFailure(code: string, message: string): ModelStreamEvent { - const normalized = code.toLowerCase(); - const rateLimited = RATE_LIMIT_CODES.has(normalized); - const overloaded = OVERLOADED_CODES.has(normalized); - return { - type: "error", - code, - message, - retryable: rateLimited || overloaded, - category: rateLimited ? "rate_limit" : overloaded ? "overloaded" : "unknown", - requestPhase: "streaming", - }; +function requirePrepared(request: PreparedModelRequest): void { + if (!isPreparedModelRequest(request)) { + throw new ResponsesCodecError("OpenAI Responses requires a prepared model request"); + } } -export class ResponsesCodecError extends Error { - constructor(message: string, options?: { cause?: unknown }) { - super(message, options); - this.name = "ResponsesCodecError"; +function preparedContent( + request: PreparedModelRequest, + content: Extract["content"], +): JsonValue[] { + return content.map((item): JsonValue => { + if (item.type === "text") return { type: "input_text", text: item.text }; + const blob = request.preparation.blobs.get(item.blob.sha256); + if (blob === undefined) { + throw new ResponsesCodecError(`Prepared blob ${item.blob.sha256} is unavailable`); + } + return { type: "input_image", detail: "auto", image_url: preparedBlobDataUrl(blob) }; + }); +} + +function plainToolOutput(parts: readonly JsonValue[]): JsonValue { + if (parts.length === 1) { + const part = object(parts[0]); + if (part?.type === "input_text" && typeof part.text === "string") return part.text; } + return parts as JsonValue; } -/** Encodes a canonical request as an OpenAI Responses API streaming body. */ -export function encodeResponsesRequest( - model: ModelInfo, - request: ModelRequest, - deployment: string, - resolvedBlobs: ReadonlyMap = new Map(), -): JsonObject { +function parseReasoningSignature(value: string, path: string): JsonObject { + try { + const parsed = JSON.parse(value) as unknown; + const item = object(parsed); + if (item?.type !== "reasoning") throw new Error("signature is not a reasoning item"); + return parsed as JsonObject; + } catch (error) { + throw new ResponsesCodecError(`${path} has an invalid Responses reasoning signature`, { + cause: error, + }); + } +} + +function fallbackMessageId(messageIndex: number, contentIndex: number): string { + return `msg_axl_${messageIndex}_${contentIndex}`; +} + +function toolForCall( + request: PreparedModelRequest, + call: { readonly canonicalName: string }, +): PreparedToolDeclaration | undefined { + return request.preparation.tools.find((tool) => tool.canonicalName === call.canonicalName); +} + +function encodeMessages(request: PreparedModelRequest): JsonValue[] { const input: JsonValue[] = []; - for (const message of request.messages) { + for (const [messageIndex, message] of request.messages.entries()) { if (message.role === "user") { + input.push({ role: "user", content: preparedContent(request, message.content) }); + continue; + } + if (message.role === "tool") { + const output = preparedContent(request, message.content); + const tool = request.preparation.tools.find( + (candidate) => candidate.canonicalName === message.canonicalName, + ); input.push({ - role: "user", - content: contentParts(message.content, "input_text", resolvedBlobs), + type: + tool?.preparedConstraint?.type === "grammar" + ? "custom_tool_call_output" + : "function_call_output", + call_id: message.callId, + output: plainToolOutput(output), }); - } else if (message.role === "assistant") { - const text = message.content - .filter((item) => item.type === "text") - .map((item) => item.text) - .join(""); - if (text.length > 0) { - input.push({ role: "assistant", content: [{ type: "output_text", text }] }); + continue; + } + + for (const [contentIndex, content] of message.content.entries()) { + if (content.type === "blob") { + throw new ResponsesCodecError( + "OpenAI Responses cannot replay image content from an assistant", + ); + } + if (content.type === "thinking") { + if (content.signature !== undefined) { + input.push( + parseReasoningSignature( + content.signature.value, + `messages[${messageIndex}].content[${contentIndex}]`, + ), + ); + } + continue; } - for (const call of message.toolCalls ?? []) { + if (content.text.length === 0) continue; + input.push({ + type: "message", + role: "assistant", + status: "completed", + id: content.continuation?.itemId ?? fallbackMessageId(messageIndex, contentIndex), + content: [{ type: "output_text", text: content.text, annotations: [] }], + }); + } + + for (const call of message.toolCalls ?? []) { + const continuation = call.continuation; + const tool = toolForCall(request, call); + if (tool?.preparedConstraint?.type === "grammar") { + const value = call.input[tool.preparedConstraint.inputProperty]; + if (typeof value !== "string") { + throw new ResponsesCodecError( + `Tool call ${call.canonicalCallId} grammar input must be a string`, + ); + } + input.push({ + type: "custom_tool_call", + call_id: call.callId, + name: call.name, + input: value, + ...(continuation?.itemId === undefined ? {} : { id: continuation.itemId }), + ...(continuation?.namespace === undefined ? {} : { namespace: continuation.namespace }), + }); + } else { input.push({ type: "function_call", call_id: call.callId, name: call.name, arguments: JSON.stringify(call.input), + ...(continuation?.itemId === undefined ? {} : { id: continuation.itemId }), + ...(continuation?.namespace === undefined ? {} : { namespace: continuation.namespace }), }); } - } else { - const output = contentParts(message.content, "input_text", resolvedBlobs); - const only = output[0]; - const plain = - output.length === 1 && - typeof only === "object" && - only !== null && - "type" in only && - only.type === "input_text" && - "text" in only && - typeof only.text === "string" - ? only.text - : undefined; - input.push({ - type: "function_call_output", - call_id: message.callId, - output: plain ?? output, - }); } } + return input; +} - const body: Record = { - model: deployment, - input, - stream: true, - store: false, - }; - if (request.system !== undefined) body.instructions = request.system; - const configuration = fitModelRequest(model, request); - if (configuration.maxOutputTokens < MIN_OUTPUT_TOKENS) - throw new ResponsesCodecError( - `OpenAI Responses needs at least ${MIN_OUTPUT_TOKENS} output tokens; the requested or available ceiling is ${configuration.maxOutputTokens}`, - ); - body.max_output_tokens = configuration.maxOutputTokens; - if (request.tools !== undefined && request.tools.length > 0) { - body.tools = request.tools.map((tool) => ({ +function encodeTools(request: PreparedModelRequest): JsonValue[] | undefined { + if (request.preparation.tools.length === 0) return undefined; + return request.preparation.tools.map((tool): JsonValue => { + const constraint = tool.preparedConstraint; + if (constraint?.type === "grammar") { + return { + type: "custom", + name: tool.name, + description: tool.description, + format: { + type: "grammar", + syntax: constraint.format, + definition: constraint.definition, + }, + }; + } + return { type: "function", name: tool.name, description: tool.description, parameters: tool.inputSchema, - strict: false, - })); - if (request.toolChoice !== undefined) body.tool_choice = request.toolChoice; - } else if (request.toolChoice === "required") { - throw new ResponsesCodecError("toolChoice required needs at least one tool"); - } - // `off` omits the reasoning parameter entirely rather than sending a zero. - const level = request.thinkingLevel; - if (model.reasoning && level !== undefined && level !== "off") { - body.reasoning = { effort: model.thinkingLevelMap?.[level] ?? level }; - } - return body; + strict: constraint?.type === "json-schema" ? constraint.strict : false, + }; + }); } -function contentParts( - content: readonly { - type: string; - text?: string; - blob?: BlobReference; - }[], - textType: "input_text", - resolvedBlobs: ReadonlyMap, -): JsonValue[] { - return content.map((item) => { - if (item.type === "text") return { type: textType, text: item.text ?? "" }; - if (item.type === "blob" && item.blob !== undefined) { - if (!item.blob.mediaType.startsWith("image/")) { - throw new ResponsesCodecError( - `OpenAI Responses does not accept attachment type ${item.blob.mediaType}`, - ); - } - const data = resolvedBlobs.get(item.blob.sha256); - if (data === undefined) { - throw new ResponsesCodecError( - `Cannot encode blob ${item.blob.sha256} without media transport`, - ); - } - return { - type: "input_image", - detail: "auto", - image_url: `data:${item.blob.mediaType};base64,${data}`, - }; +function applySampling(body: MutableJsonObject, request: PreparedModelRequest): void { + if (request.sampling?.temperature !== undefined) body.temperature = request.sampling.temperature; + if (request.sampling?.topP !== undefined) body.top_p = request.sampling.topP; + for (const [field, value] of Object.entries(request.sampling?.custom ?? {})) { + if (RESERVED_REQUEST_FIELDS.has(field) || field in body) { + throw new ResponsesCodecError(`Custom sampling field ${field} collides with a request field`); } - throw new ResponsesCodecError(`Cannot encode ${item.type} content`); - }); + body[field] = value; + } } -async function resolveRequestBlobs(request: ModelRequest): Promise> { - const references = new Map(); - for (const message of request.messages) { - for (const item of message.content) { - if (item.type === "blob") references.set(item.blob.sha256, item.blob); +function applyCache( + body: MutableJsonObject, + headers: Record, + model: ModelInfo, + request: PreparedModelRequest, +): void { + const cache = request.preparation.cache; + if (cache.retention === "none") return; + const compat = compatibility(model); + if (cache.retention === "long") { + if (compat.supportsLongCacheRetention !== true) { + throw new ResponsesCodecError( + "Long cache retention is unsupported by this Responses endpoint", + ); } + body.prompt_cache_retention = "24h"; + } + if (cache.sessionId === undefined) return; + body.prompt_cache_key = Array.from(cache.sessionId).slice(0, 64).join(""); + if (compat.sessionAffinityFormat === "openrouter") { + headers["x-session-id"] = cache.sessionId; + } else if (compat.sessionAffinityFormat === "openai") { + headers.session_id = cache.sessionId; + headers["x-client-request-id"] = cache.sessionId; + headers["x-session-affinity"] = cache.sessionId; + } else if (compat.sessionAffinityFormat === "openai-no-session") { + headers["x-client-request-id"] = cache.sessionId; + headers["x-session-affinity"] = cache.sessionId; + } +} + +/** Encodes only a validated, immutable prepared request. */ +export function encodeResponsesRequest( + model: ModelInfo, + request: PreparedModelRequest, + wireModelId = model.modelId, +): EncodedResponsesRequest { + requirePrepared(request); + const compat = compatibility(model); + if (request.modelId !== model.modelId) { + throw new ResponsesCodecError( + `Request model ${request.modelId} does not match ${model.modelId}`, + ); + } + if (request.metadata !== undefined && Object.keys(request.metadata).length > 0) { + throw new ResponsesCodecError("OpenAI Responses cannot render request metadata"); } - if (references.size === 0) return new Map(); - if (request.readBlob === undefined) { - throw new ResponsesCodecError("Cannot encode blob content without media transport"); + + const body: MutableJsonObject = { + model: wireModelId, + input: encodeMessages(request), + stream: true, + store: false, + }; + if (request.system !== undefined) body.instructions = request.system; + if (request.maxOutputTokens !== undefined) { + if (compat.supportsMaxOutputTokens === false) { + throw new ResponsesCodecError("This Responses endpoint does not support max_output_tokens"); + } + body.max_output_tokens = Math.max(request.maxOutputTokens, MIN_OUTPUT_TOKENS); } - const resolved = new Map(); - for (const reference of references.values()) { - const bytes = await request.readBlob(reference); - if (bytes.byteLength !== reference.sizeBytes) { - throw new ResponsesCodecError(`Blob ${reference.sha256} size changed before dispatch`); + const tools = encodeTools(request); + if (tools !== undefined) body.tools = tools; + if (request.toolChoice !== undefined) { + if (request.toolChoice !== "none" && tools === undefined) { + throw new ResponsesCodecError(`toolChoice ${request.toolChoice} needs at least one tool`); } - resolved.set(reference.sha256, Buffer.from(bytes).toString("base64")); + body.tool_choice = request.toolChoice; + } + const reasoning = request.preparation.reasoning; + if (reasoning !== undefined && reasoning.effective !== "off") { + body.reasoning = { effort: reasoning.providerValue ?? reasoning.effective, summary: "auto" }; + body.include = ["reasoning.encrypted_content"]; } - return resolved; + applySampling(body, request); + const headers: Record = {}; + applyCache(body, headers, model, request); + return { body, headers }; } -function mapUsage(raw: Record | undefined): Usage { - const usage = (raw ?? {}) as { - input_tokens?: number; - output_tokens?: number; - input_tokens_details?: { cached_tokens?: number; cache_write_tokens?: number }; - output_tokens_details?: { reasoning_tokens?: number }; - }; - const cached = usage.input_tokens_details?.cached_tokens ?? 0; - const cacheWrite = usage.input_tokens_details?.cache_write_tokens ?? 0; - return { - // The API includes cached and cache-write tokens in input_tokens; subtract both. - inputTokens: Math.max(0, (usage.input_tokens ?? 0) - cached - cacheWrite), - outputTokens: usage.output_tokens ?? 0, +function mapUsage(raw: unknown, model: ModelInfo, includeCost: boolean): Usage { + const value = object(raw) ?? {}; + const inputDetails = object(value.input_tokens_details); + const outputDetails = object(value.output_tokens_details); + const input = typeof value.input_tokens === "number" ? value.input_tokens : 0; + const cached = typeof inputDetails?.cached_tokens === "number" ? inputDetails.cached_tokens : 0; + const cacheWrite = + typeof inputDetails?.cache_write_tokens === "number" ? inputDetails.cache_write_tokens : 0; + const mapped: Usage = { + inputTokens: Math.max(0, input - cached - cacheWrite), + outputTokens: typeof value.output_tokens === "number" ? value.output_tokens : 0, cacheReadTokens: cached, cacheWriteTokens: cacheWrite, - reasoningTokens: usage.output_tokens_details?.reasoning_tokens ?? 0, + reasoningTokens: + typeof outputDetails?.reasoning_tokens === "number" ? outputDetails.reasoning_tokens : 0, }; + return !includeCost || model.cost === undefined ? mapped : withUsageCost(model.cost, mapped); +} + +function providerErrorCategory(code: string): ModelErrorCategory { + const normalized = code.toLowerCase(); + if (RATE_LIMIT_CODES.has(normalized)) return "rate_limit"; + if (OVERLOADED_CODES.has(normalized)) return "overloaded"; + if (normalized === "timeout") return "timeout"; + return "unknown"; +} + +function retryAfterMs(headers: Headers, now = Date.now()): number | undefined { + const value = headers.get("retry-after")?.trim(); + if (!value) return undefined; + const seconds = Number(value); + if (Number.isFinite(seconds) && seconds >= 0) return Math.round(seconds * 1_000); + const date = Date.parse(value); + return Number.isFinite(date) ? Math.max(0, date - now) : undefined; +} + +function nestedErrorCode(error: unknown): string | undefined { + let current = error; + for (let depth = 0; depth < 4; depth += 1) { + if (typeof current !== "object" || current === null) return undefined; + const candidate = current as { code?: unknown; cause?: unknown }; + if (typeof candidate.code === "string") return candidate.code; + current = candidate.cause; + } + return undefined; +} + +interface OutputSlot { + readonly type: "thinking" | "text" | "function" | "custom"; + readonly contentIndex: number; + readonly callId?: string; + readonly name?: string; + arguments: string; + started: boolean; } -/** - * Decodes Responses API SSE frames into canonical stream events. Ends after - * the terminal event; a stream that ends without one simply returns, and - * `normalizeModelStream` converts that into an error terminal. - */ +export interface ResponsesDecodeOptions { + readonly model: ModelInfo; + readonly request: PreparedModelRequest; + readonly startedAtMs?: number; + readonly now?: () => number; + readonly secretValues?: readonly string[]; + /** Transitional transports can retain their pre-codec usage shape. */ + readonly includeCost?: boolean; +} + +/** Decodes Responses API SSE frames into canonical stream events. */ export async function* decodeResponsesStream( frames: AsyncIterable, + options: ResponsesDecodeOptions, ): AsyncGenerator { - const calls = new Map(); + requirePrepared(options.request); + compatibility(options.model); + const slots = new Map(); + let responseId: string | undefined; + let routedModelId: string | undefined; + let emittedContent = false; let sawToolCall = false; + const metadata = (nativeStopReason?: string) => ({ + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined ? {} : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + ...(nativeStopReason === undefined ? {} : { nativeStopReason }), + ...(options.startedAtMs === undefined + ? {} + : { latencyMs: Math.max(0, (options.now ?? Date.now)() - options.startedAtMs) }), + }); + + const replay = ( + target: "thinking" | "text" | "tool_call", + contentIndex: number, + data: { callId?: string; signature?: string; itemId?: string; namespace?: string }, + ): ModelStreamEvent => ({ + type: "replay_metadata", + target, + contentIndex, + providerId: options.model.providerId, + apiDialect: options.model.apiDialect, + modelId: options.model.modelId, + ...(data.callId === undefined ? {} : { callId: data.callId }), + ...(data.signature === undefined ? {} : { signature: data.signature }), + ...(responseId === undefined ? {} : { responseId }), + ...(data.itemId === undefined ? {} : { itemId: data.itemId }), + ...(data.namespace === undefined ? {} : { namespace: data.namespace }), + }); + for await (const frame of frames) { + if (options.request.signal?.aborted) { + yield { type: "aborted", ...(emittedContent ? { partial: true } : {}) }; + return; + } if (frame.data === "[DONE]") continue; let event: Record; try { - event = JSON.parse(frame.data) as Record; + const parsed = JSON.parse(frame.data) as unknown; + const parsedObject = object(parsed); + if (parsedObject === undefined) throw new Error("frame is not an object"); + event = parsedObject; } catch (error) { - throw new ResponsesCodecError("Provider sent an undecodable stream frame", { cause: error }); + throw new ResponsesCodecError("Provider sent an undecodable Responses stream frame", { + cause: error, + }); } const type = event.type; - if (type === "response.output_text.delta") { - yield { type: "text_delta", text: String(event.delta ?? "") }; - } else if ( + if (type === "response.created") { + const response = object(event.response); + if (typeof response?.id === "string" && response.id.length > 0) responseId = response.id; + if (typeof response?.model === "string" && response.model.length > 0) + routedModelId = response.model; + continue; + } + + if (type === "response.output_item.added") { + const outputIndex = event.output_index; + const item = object(event.item); + if (!Number.isSafeInteger(outputIndex) || (outputIndex as number) < 0 || item === undefined) { + throw new ResponsesCodecError("Provider sent an output item without a valid index"); + } + const index = outputIndex as number; + if (slots.has(index)) throw new ResponsesCodecError(`Provider reused output index ${index}`); + if (item.type === "reasoning") { + slots.set(index, { type: "thinking", contentIndex: index, arguments: "", started: true }); + } else if (item.type === "message") { + slots.set(index, { type: "text", contentIndex: index, arguments: "", started: true }); + } else if (item.type === "function_call" || item.type === "custom_tool_call") { + const callId = typeof item.call_id === "string" ? item.call_id : ""; + const name = typeof item.name === "string" ? item.name : ""; + if (callId.length === 0 || name.length === 0) { + throw new ResponsesCodecError("Provider sent a tool call without an id or name"); + } + const canonicalName = + options.request.preparation.tools.find((tool) => tool.name === name)?.canonicalName ?? + name; + slots.set(index, { + type: item.type === "function_call" ? "function" : "custom", + contentIndex: index, + callId, + name: canonicalName, + arguments: + item.type === "function_call" && typeof item.arguments === "string" + ? item.arguments + : item.type === "custom_tool_call" && typeof item.input === "string" + ? item.input + : "", + started: true, + }); + emittedContent = true; + yield { type: "tool_call_start", contentIndex: index, callId, name: canonicalName }; + } + continue; + } + + if ( type === "response.reasoning_text.delta" || type === "response.reasoning_summary_text.delta" ) { - yield { type: "thinking_delta", text: String(event.delta ?? "") }; - } else if (type === "response.output_item.added") { - const item = event.item as { type?: string; call_id?: string; name?: string } | undefined; - if (item?.type === "function_call") { - calls.set(Number(event.output_index ?? 0), { - callId: String(item.call_id ?? ""), - name: String(item.name ?? ""), - args: "", - }); + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (typeof index === "number" && slot?.type !== "thinking") { + throw new ResponsesCodecError("Reasoning delta has no reasoning item"); } - } else if (type === "response.function_call_arguments.delta") { - const call = calls.get(Number(event.output_index ?? 0)); - if (call) call.args += String(event.delta ?? ""); - } else if (type === "response.function_call_arguments.done") { - const call = calls.get(Number(event.output_index ?? 0)); - if (call && typeof event.arguments === "string") call.args = event.arguments; - } else if (type === "response.output_item.done") { - const call = calls.get(Number(event.output_index ?? 0)); - if (call !== undefined) { - calls.delete(Number(event.output_index ?? 0)); - let inputValue: unknown; - try { - inputValue = call.args === "" ? {} : JSON.parse(call.args); - } catch (error) { - throw new ResponsesCodecError(`Tool call ${call.callId} has undecodable arguments`, { - cause: error, - }); - } - if (call.callId.length === 0 || call.name.length === 0) { - throw new ResponsesCodecError("Provider sent a tool call without an id or name"); + const delta = typeof event.delta === "string" ? event.delta : ""; + if (delta.length > 0) emittedContent = true; + yield { + type: "thinking_delta", + text: delta, + ...(slot === undefined ? {} : { contentIndex: slot.contentIndex }), + }; + continue; + } + + if (type === "response.reasoning_summary_part.done") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot?.type !== "thinking") + throw new ResponsesCodecError("Reasoning part has no reasoning item"); + emittedContent = true; + yield { type: "thinking_delta", text: "\n\n", contentIndex: slot.contentIndex }; + continue; + } + + if (type === "response.output_text.delta" || type === "response.refusal.delta") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (typeof index === "number" && slot?.type !== "text") { + throw new ResponsesCodecError("Text delta has no message item"); + } + const delta = typeof event.delta === "string" ? event.delta : ""; + if (delta.length > 0) emittedContent = true; + yield { + type: "text_delta", + text: delta, + ...(slot === undefined ? {} : { contentIndex: slot.contentIndex }), + }; + continue; + } + + if (type === "response.function_call_arguments.delta") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot?.type !== "function" || slot.callId === undefined) { + throw new ResponsesCodecError("Function arguments delta has no function call item"); + } + const delta = typeof event.delta === "string" ? event.delta : ""; + slot.arguments += delta; + emittedContent = true; + yield { + type: "tool_call_delta", + contentIndex: slot.contentIndex, + callId: slot.callId, + argumentsDelta: delta, + }; + continue; + } + + if (type === "response.function_call_arguments.done") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot?.type !== "function") { + throw new ResponsesCodecError("Function arguments completion has no function call item"); + } + if (typeof event.arguments === "string") slot.arguments = event.arguments; + continue; + } + + if (type === "response.custom_tool_call_input.delta") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot?.type !== "custom" || slot.callId === undefined) { + throw new ResponsesCodecError("Custom tool delta has no custom tool item"); + } + const delta = typeof event.delta === "string" ? event.delta : ""; + slot.arguments += delta; + const tool = options.request.preparation.tools.find( + (candidate) => candidate.name === slot.name, + ); + const property = + tool?.preparedConstraint?.type === "grammar" + ? tool.preparedConstraint.inputProperty + : "input"; + emittedContent = true; + yield { + type: "tool_call_delta", + contentIndex: slot.contentIndex, + callId: slot.callId, + argumentsDelta: JSON.stringify({ [property]: slot.arguments }), + }; + continue; + } + + if (type === "response.custom_tool_call_input.done") { + const index = event.output_index; + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot?.type !== "custom") { + throw new ResponsesCodecError("Custom tool completion has no custom tool item"); + } + if (typeof event.input === "string") slot.arguments = event.input; + continue; + } + + if (type === "response.output_item.done") { + const index = event.output_index; + const item = object(event.item); + const slot = typeof index === "number" ? slots.get(index) : undefined; + if (slot === undefined || item === undefined) { + throw new ResponsesCodecError("Completed output item has no matching item"); + } + slots.delete(index as number); + const itemId = typeof item.id === "string" && item.id.length > 0 ? item.id : undefined; + const namespace = + typeof item.namespace === "string" && item.namespace.length > 0 + ? item.namespace + : undefined; + if (slot.type === "thinking") { + const signature = JSON.stringify(item); + yield replay("thinking", slot.contentIndex, { + signature, + ...(itemId === undefined ? {} : { itemId }), + }); + } else if (slot.type === "text") { + yield replay("text", slot.contentIndex, { + ...(itemId === undefined ? {} : { itemId }), + ...(typeof item.phase === "string" ? { namespace: item.phase } : {}), + }); + } else { + if (slot.callId === undefined || slot.name === undefined) { + throw new ResponsesCodecError("Completed tool item is missing its identity"); } - if (typeof inputValue !== "object" || inputValue === null || Array.isArray(inputValue)) { - throw new ResponsesCodecError(`Tool call ${call.callId} arguments must be an object`); + let input: JsonObject; + if (slot.type === "custom") { + const tool = options.request.preparation.tools.find( + (candidate) => candidate.canonicalName === slot.name || candidate.name === slot.name, + ); + const property = + tool?.preparedConstraint?.type === "grammar" + ? tool.preparedConstraint.inputProperty + : "input"; + input = { [property]: typeof item.input === "string" ? item.input : slot.arguments }; + } else { + const source = + typeof item.arguments === "string" ? item.arguments : slot.arguments || "{}"; + try { + const parsed = JSON.parse(source) as unknown; + if (object(parsed) === undefined) throw new Error("arguments are not an object"); + input = parsed as JsonObject; + } catch (error) { + throw new ResponsesCodecError(`Tool call ${slot.callId} has undecodable arguments`, { + cause: error, + }); + } } sawToolCall = true; yield { type: "tool_call", - callId: call.callId, - name: call.name, - input: inputValue as JsonObject, + contentIndex: slot.contentIndex, + callId: slot.callId, + name: slot.name, + input, + }; + if (itemId !== undefined || namespace !== undefined || responseId !== undefined) { + yield replay("tool_call", slot.contentIndex, { + callId: slot.callId, + ...(itemId === undefined ? {} : { itemId }), + ...(namespace === undefined ? {} : { namespace }), + }); + } + } + continue; + } + + if (type === "response.completed" || type === "response.incomplete") { + const response = object(event.response); + if (typeof response?.id === "string" && response.id.length > 0) responseId = response.id; + if (typeof response?.model === "string" && response.model.length > 0) + routedModelId = response.model; + const details = object(response?.incomplete_details); + const reason = typeof details?.reason === "string" ? details.reason : undefined; + const status = typeof response?.status === "string" ? response.status : undefined; + const nativeStopReason = reason ?? status; + if ( + type === "response.incomplete" && + reason !== undefined && + reason !== "max_output_tokens" + ) { + yield { + type: "error", + code: "response_incomplete", + message: `Response incomplete: ${reason}`, + retryable: false, + ...(emittedContent ? { partial: true } : {}), + response: metadata(nativeStopReason), }; + return; } - } else if (type === "response.completed" || type === "response.incomplete") { - const response = event.response as { usage?: Record } | undefined; yield { type: "completed", stopReason: type === "response.incomplete" ? "length" : sawToolCall ? "tool_use" : "stop", - usage: mapUsage(response?.usage), + usage: mapUsage(response?.usage, options.model, options.includeCost !== false), + ...(type === "response.incomplete" ? { partial: true } : {}), + response: metadata(nativeStopReason), }; return; - } else if (type === "response.failed" || type === "error") { - const response = event.response as - | { error?: { message?: string; code?: string } } - | undefined; - const message = response?.error?.message ?? (event.message as string | undefined); - yield providerFailure( - String(response?.error?.code ?? event.code ?? "provider_error"), - message ?? "Provider reported a failure", - ); + } + + if (type === "response.failed" || type === "error") { + const response = object(event.response); + const providerError = object(response?.error) ?? object(event.error); + const code = String(providerError?.code ?? event.code ?? "provider_error"); + const rawMessage = + typeof providerError?.message === "string" + ? providerError.message + : typeof event.message === "string" + ? event.message + : "Provider reported a failure"; + const category = providerErrorCategory(code); + yield { + type: "error", + code, + message: safeProviderMessage(rawMessage, options.secretValues), + retryable: category === "rate_limit" || category === "overloaded" || category === "timeout", + category, + requestPhase: "streaming", + ...(emittedContent ? { partial: true } : {}), + response: metadata(typeof response?.status === "string" ? response.status : undefined), + }; return; } - // Unknown event types are forward-compatible noise and are ignored. + // Unknown top-level event types are forward-compatible noise. } } @@ -356,23 +794,19 @@ export interface OpenAiResponsesProviderOptions { readonly id: string; readonly displayName: string; readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; readonly endpoint: ResponsesEndpoint; readonly models: readonly ModelInfo[]; readonly resolveAuth: () => Promise; readonly fetch?: typeof fetch; } -/** - * Generic OpenAI-Responses provider: composes an endpoint policy, an auth - * resolver, and an injectable fetch around the pure codec. Azure is one - * endpoint policy; any Responses-compatible host is another. Model lookup and - * capability checks fail before dispatch; every post-dispatch failure - * terminates through the stream contract. - */ +/** Legacy transport composition retained until provider registration owns transport. */ export class OpenAiResponsesProvider implements ModelProvider { readonly id: string; readonly displayName: string; readonly authMethods: readonly AuthMethod[]; + readonly authentication?: ProviderAuthentication; private readonly endpoint: ResponsesEndpoint; private readonly models: readonly ModelInfo[]; private readonly resolveAuth: () => Promise; @@ -382,6 +816,7 @@ export class OpenAiResponsesProvider implements ModelProvider { this.id = options.id; this.displayName = options.displayName; this.authMethods = options.authMethods; + if (options.authentication !== undefined) this.authentication = options.authentication; this.endpoint = options.endpoint; this.models = options.models; this.resolveAuth = options.resolveAuth; @@ -412,29 +847,41 @@ export class OpenAiResponsesProvider implements ModelProvider { body: string; signal?: AbortSignal; }; + let prepared: PreparedModelRequest; + let secretValues: readonly string[] = []; try { + prepared = isPreparedModelRequest(request) + ? request + : await prepareModelRequest(model, request); const resolved = await this.resolveAuth(); - const body = encodeResponsesRequest( + secretValues = resolved.secretValues; + registerResolvedSecrets(secretValues); + const encoded = encodeResponsesRequest( model, - request, + prepared, this.endpoint.deploymentFor(model.modelId, resolved), - await resolveRequestBlobs(request), ); - url = this.endpoint.url(resolved); + url = safeEndpoint(this.endpoint.url(resolved), { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: true, + allowQuery: true, + }); init = { method: "POST", headers: { "content-type": "application/json", accept: "text/event-stream", + ...encoded.headers, ...this.endpoint.headers(resolved), }, - body: JSON.stringify(body), + body: JSON.stringify(encoded.body), ...(request.signal === undefined ? {} : { signal: request.signal }), }; } catch (error) { yield this.failure( request, error, + secretValues, "provider_request_setup_failed", "before_dispatch", false, @@ -449,19 +896,20 @@ export class OpenAiResponsesProvider implements ModelProvider { let response: Pick; try { - response = - this.fetchImpl === undefined - ? await modelFetch(url, { - ...init, - dispatcher: dispatcherFor(fitModelRequest(model, request).httpIdleTimeoutMs), - }) - : await this.fetchImpl(url, init); + response = await safeFetch(url, init, { + label: `Provider ${this.id} request endpoint`, + allowLoopbackHttp: true, + expectedOrigin: new URL(url).origin, + idleTimeoutMs: request.httpIdleTimeoutMs ?? 300_000, + ...(this.fetchImpl === undefined ? {} : { fetch: this.fetchImpl }), + }); } catch (error) { - const code = nestedErrorCode(error); - const safeToRetry = code !== undefined && SAFE_CONNECT_FAILURES.has(code); + const nativeCode = nestedErrorCode(error); + const safeToRetry = nativeCode !== undefined && SAFE_CONNECT_FAILURES.has(nativeCode); yield this.failure( request, error, + secretValues, "provider_request_failed", safeToRetry ? "before_dispatch" : "unknown", safeToRetry, @@ -507,11 +955,17 @@ export class OpenAiResponsesProvider implements ModelProvider { } try { - yield* decodeResponsesStream(decodeSseStream(response.body)); + yield* decodeResponsesStream(decodeSseStream(response.body), { + model, + request: prepared, + secretValues, + includeCost: false, + }); } catch (error) { yield this.failure( request, error, + secretValues, "provider_stream_failed", "streaming", false, @@ -523,6 +977,7 @@ export class OpenAiResponsesProvider implements ModelProvider { private failure( request: ModelRequest, error: unknown, + secretValues: readonly string[], code: string, requestPhase: "before_dispatch" | "awaiting_response" | "streaming" | "unknown", retryable: boolean, @@ -534,17 +989,19 @@ export class OpenAiResponsesProvider implements ModelProvider { return { type: "error", code: "model_request_idle_timeout", - message: `Provider transport was idle for ${request.httpIdleTimeoutMs ?? 300_000} ms while ${transportCode === "UND_ERR_HEADERS_TIMEOUT" ? "waiting for response headers" : "reading the response body"}`, + message: `Provider ${this.id} produced no HTTP data before the configured idle timeout`, retryable: false, category: "timeout", - requestPhase: - transportCode === "UND_ERR_HEADERS_TIMEOUT" ? "awaiting_response" : "streaming", + requestPhase: requestPhase === "streaming" ? "streaming" : "awaiting_response", }; } return { type: "error", code, - message: error instanceof Error ? error.message : "provider request failed", + message: safeProviderMessage( + error instanceof Error ? error.message : "provider request failed", + secretValues, + ), retryable, category, requestPhase, diff --git a/packages/ai/src/openrouter-images.ts b/packages/ai/src/openrouter-images.ts new file mode 100644 index 0000000..7f29297 --- /dev/null +++ b/packages/ai/src/openrouter-images.ts @@ -0,0 +1,515 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +// Transport-neutral codec for OpenRouter's buffered image generation API. + +import { createHash } from "node:crypto"; + +import { + parseBlobReference, + type BlobReference, + type JsonObject, + type JsonValue, + type ModelErrorCategory, + type Usage, +} from "@axl/protocol"; + +import { safeProviderMessage } from "./diagnostics.ts"; +import type { ImageGenerationRequest, ImageGenerationResult, ImageModelInfo } from "./model.ts"; +import { withUsageCost } from "./usage.ts"; + +const OPENROUTER_IMAGE_DIALECT = "openrouter-images"; +const MAX_INPUT_REFERENCES = 16; +const MAX_OUTPUT_IMAGES = 10; +const MAX_IMAGE_PROMPT_BYTES = 1024 * 1024; +const MAX_ENCODED_IMAGE_BYTES = 8 * 1024 * 1024; +const MAX_DECODED_IMAGE_BYTES = 6 * 1024 * 1024; +const MAX_TOTAL_DECODED_IMAGE_BYTES = 16 * 1024 * 1024; +const MAX_RESPONSE_TEXT_BYTES = 4_096; +const ASPECT_RATIOS = new Set([ + "1:1", + "1:2", + "1:4", + "1:8", + "2:1", + "2:3", + "3:2", + "3:4", + "4:1", + "4:3", + "4:5", + "5:4", + "8:1", + "9:16", + "16:9", + "9:19.5", + "19.5:9", + "9:20", + "20:9", + "9:21", + "21:9", + "auto", +]); + +export class OpenRouterImageCodecError extends Error { + readonly code: string; + readonly category: ModelErrorCategory; + readonly retryable: boolean; + readonly aborted: boolean; + + constructor( + code: string, + message: string, + options: { + readonly category?: ModelErrorCategory; + readonly retryable?: boolean; + readonly aborted?: boolean; + } = {}, + ) { + super(message); + this.name = "OpenRouterImageCodecError"; + this.code = code; + this.category = options.category ?? "provider_internal"; + this.retryable = options.retryable ?? false; + this.aborted = options.aborted ?? false; + } +} + +export interface EncodedOpenRouterImageRequest { + readonly body: JsonObject; +} + +export interface OpenRouterImageDecodeOptions { + readonly model: ImageModelInfo; + readonly request: ImageGenerationRequest; + readonly secretValues?: readonly string[]; +} + +function object(value: unknown): Record | undefined { + return typeof value === "object" && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function inputError(message: string): never { + throw new OpenRouterImageCodecError("invalid_image_request", message, { + category: "invalid_request", + }); +} + +function responseError(message: string): never { + throw new OpenRouterImageCodecError("malformed_image_response", message); +} + +function checkCancellation(signal: AbortSignal | undefined): void { + if (signal?.aborted) { + throw new OpenRouterImageCodecError("request_aborted", "Image generation was cancelled", { + aborted: true, + }); + } +} + +function validateModel(model: ImageModelInfo, request: ImageGenerationRequest): void { + if (model.apiDialect !== OPENROUTER_IMAGE_DIALECT) { + inputError(`Model ${model.modelId} does not use the OpenRouter image dialect`); + } + if (model.providerId !== "openrouter") { + inputError(`Model ${model.modelId} is not owned by OpenRouter`); + } + if (request.modelId !== model.modelId) { + inputError(`Request model ${request.modelId} does not match ${model.modelId}`); + } + if (!model.input.includes("text") || !model.output.includes("image")) { + inputError(`Model ${model.modelId} does not support text to image generation`); + } + if (model.availability?.status === "unavailable") { + inputError(`Model ${model.modelId} is unavailable`); + } +} + +function validateRequest(request: ImageGenerationRequest): void { + if ( + typeof request.prompt !== "string" || + request.prompt.trim().length === 0 || + Buffer.byteLength(request.prompt) > MAX_IMAGE_PROMPT_BYTES + ) { + inputError("Image prompt must be non-empty and within its byte limit"); + } + if ( + request.count !== undefined && + (!Number.isSafeInteger(request.count) || request.count < 1 || request.count > MAX_OUTPUT_IMAGES) + ) { + inputError(`Image count must be an integer from 1 to ${MAX_OUTPUT_IMAGES}`); + } + if (request.size !== undefined) { + if ( + !Number.isSafeInteger(request.size.width) || + request.size.width <= 0 || + !Number.isSafeInteger(request.size.height) || + request.size.height <= 0 + ) { + inputError("Image size must contain positive safe integer dimensions"); + } + } + if (request.aspectRatio !== undefined && !ASPECT_RATIOS.has(request.aspectRatio)) { + inputError(`Image aspect ratio ${request.aspectRatio} is unsupported`); + } + if ( + request.size !== undefined && + request.aspectRatio !== undefined && + request.aspectRatio !== "auto" + ) { + const [width, height] = request.aspectRatio.split(":").map(Number); + const requestedRatio = request.size.width / request.size.height; + if ( + width === undefined || + height === undefined || + Math.abs(requestedRatio - width / height) > 1e-9 + ) { + inputError("Image size and aspect ratio are inconsistent"); + } + } + if (request.inputImages !== undefined && request.inputImages.length > MAX_INPUT_REFERENCES) { + inputError(`OpenRouter accepts at most ${MAX_INPUT_REFERENCES} input images`); + } + if (request.metadata !== undefined) { + inputError("OpenRouter image generation cannot render request metadata"); + } + if (typeof request.writeBlob !== "function") inputError("Image blob writer must be a function"); +} + +function digest(bytes: Uint8Array): string { + return createHash("sha256").update(bytes).digest("hex"); +} + +async function encodeInputReferences( + model: ImageModelInfo, + request: ImageGenerationRequest, +): Promise { + const references = request.inputImages; + if (references === undefined || references.length === 0) return undefined; + if (!model.input.includes("image")) + inputError(`Model ${model.modelId} does not support image input`); + if (request.readBlob === undefined) inputError("Image blob reader is required for input images"); + + const encoded: JsonValue[] = []; + for (const [index, rawReference] of references.entries()) { + checkCancellation(request.signal); + let reference: BlobReference; + try { + reference = parseBlobReference(rawReference, `request.inputImages[${index}]`); + } catch (error) { + inputError(error instanceof Error ? error.message : `Input image ${index} is malformed`); + } + if (!reference.mediaType.startsWith("image/")) { + inputError(`Input image ${index} must have an image media type`); + } + const bytes = await request.readBlob(reference); + checkCancellation(request.signal); + if (!(bytes instanceof Uint8Array)) inputError(`Input image ${index} loader must return bytes`); + if (bytes.byteLength !== reference.sizeBytes || digest(bytes) !== reference.sha256) { + inputError(`Input image ${index} does not match its content address`); + } + encoded.push({ + type: "image_url", + image_url: { + url: `data:${reference.mediaType};base64,${Buffer.from(bytes).toString("base64")}`, + }, + }); + } + return encoded; +} + +/** Encodes one validated request for the buffered OpenRouter image endpoint. */ +export async function encodeOpenRouterImageRequest( + model: ImageModelInfo, + request: ImageGenerationRequest, +): Promise { + checkCancellation(request.signal); + validateModel(model, request); + validateRequest(request); + const inputReferences = await encodeInputReferences(model, request); + const body: Record = { + model: model.modelId, + prompt: request.prompt, + }; + if (request.count !== undefined) body.n = request.count; + if (request.size !== undefined) body.size = `${request.size.width}x${request.size.height}`; + if (request.aspectRatio !== undefined) body.aspect_ratio = request.aspectRatio; + if (inputReferences !== undefined) body.input_references = inputReferences; + return { body }; +} + +function nonNegativeInteger(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) < 0) { + responseError(`${label} must be a non-negative safe integer`); + } + return value as number; +} + +function nonNegativeNumber(value: unknown, label: string): number { + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + responseError(`${label} must be a non-negative number`); + } + return value; +} + +function parseUsage(raw: unknown, model: ImageModelInfo): Usage | undefined { + if (raw === undefined) return undefined; + const value = object(raw); + if (value === undefined) responseError("OpenRouter image usage must be an object"); + const prompt = + value.prompt_tokens === undefined + ? 0 + : nonNegativeInteger(value.prompt_tokens, "OpenRouter prompt tokens"); + const output = + value.completion_tokens === undefined + ? 0 + : nonNegativeInteger(value.completion_tokens, "OpenRouter completion tokens"); + const promptDetails = object(value.prompt_tokens_details); + if (value.prompt_tokens_details !== undefined && promptDetails === undefined) { + responseError("OpenRouter prompt token details must be an object"); + } + const completionDetails = object(value.completion_tokens_details); + if (value.completion_tokens_details !== undefined && completionDetails === undefined) { + responseError("OpenRouter completion token details must be an object"); + } + const reportedCached = + promptDetails?.cached_tokens === undefined + ? 0 + : nonNegativeInteger(promptDetails.cached_tokens, "OpenRouter cached tokens"); + const cacheWrite = + promptDetails?.cache_write_tokens === undefined + ? 0 + : nonNegativeInteger(promptDetails.cache_write_tokens, "OpenRouter cache write tokens"); + const cacheRead = cacheWrite > 0 ? Math.max(0, reportedCached - cacheWrite) : reportedCached; + const reasoning = + completionDetails?.reasoning_tokens === undefined + ? 0 + : nonNegativeInteger(completionDetails.reasoning_tokens, "OpenRouter reasoning tokens"); + if (value.total_tokens !== undefined) { + nonNegativeInteger(value.total_tokens, "OpenRouter total tokens"); + } + const usage: Usage = { + inputTokens: Math.max(0, prompt - cacheRead - cacheWrite), + outputTokens: output, + cacheReadTokens: cacheRead, + cacheWriteTokens: cacheWrite, + reasoningTokens: reasoning, + }; + if (value.cost !== undefined) { + return { ...usage, costUsd: nonNegativeNumber(value.cost, "OpenRouter image cost") }; + } + return model.cost === undefined ? usage : withUsageCost(model.cost, usage); +} + +function stripBase64Padding(value: string): string { + let end = value.length; + while (end > 0 && value.charCodeAt(end - 1) === 61) end -= 1; + return value.slice(0, end); +} + +function decodeBase64(value: unknown, index: number): Uint8Array { + if (typeof value !== "string" || value.length === 0 || value.length > MAX_ENCODED_IMAGE_BYTES) { + responseError(`OpenRouter image ${index} has no bounded base64 data`); + } + if (!/^[A-Za-z0-9+/]*={0,2}$/.test(value) || value.length % 4 === 1) { + responseError(`OpenRouter image ${index} has malformed base64 data`); + } + const bytes = new Uint8Array(Buffer.from(value, "base64")); + if (bytes.length === 0 || bytes.length > MAX_DECODED_IMAGE_BYTES) + responseError(`OpenRouter image ${index} decoded outside its byte limit`); + const normalizedInput = stripBase64Padding(value); + const normalizedOutput = stripBase64Padding(Buffer.from(bytes).toString("base64")); + if (normalizedInput !== normalizedOutput) { + responseError(`OpenRouter image ${index} has malformed base64 data`); + } + return bytes; +} + +function inferredMediaType(bytes: Uint8Array): string | undefined { + if ( + bytes.length >= 8 && + bytes[0] === 0x89 && + bytes[1] === 0x50 && + bytes[2] === 0x4e && + bytes[3] === 0x47 && + bytes[4] === 0x0d && + bytes[5] === 0x0a && + bytes[6] === 0x1a && + bytes[7] === 0x0a + ) { + return "image/png"; + } + if (bytes.length >= 3 && bytes[0] === 0xff && bytes[1] === 0xd8 && bytes[2] === 0xff) { + return "image/jpeg"; + } + const prefix = Buffer.from(bytes.subarray(0, 16)).toString("ascii"); + if (prefix.startsWith("GIF87a") || prefix.startsWith("GIF89a")) return "image/gif"; + if (prefix.startsWith("RIFF") && prefix.slice(8, 12) === "WEBP") return "image/webp"; + const textPrefix = Buffer.from(bytes.subarray(0, 256)).toString("utf8").trimStart(); + if (textPrefix.startsWith("]*>\s*, + secretValues: readonly string[] | undefined, +): never { + const details = object(raw.error); + const rawCode = details?.code ?? details?.type ?? raw.code ?? "openrouter_image_error"; + const code = + typeof rawCode === "string" && rawCode.length > 0 ? rawCode : "openrouter_image_error"; + const rawMessage = + typeof details?.message === "string" + ? details.message + : typeof raw.error === "string" + ? raw.error + : typeof raw.message === "string" + ? raw.message + : "OpenRouter image generation failed"; + const normalizedCode = code.toLowerCase(); + const category: ModelErrorCategory = normalizedCode.includes("rate") + ? "rate_limit" + : normalizedCode.includes("overload") + ? "overloaded" + : normalizedCode.includes("auth") + ? "authentication" + : normalizedCode.includes("permission") || normalizedCode.includes("forbidden") + ? "authorization" + : normalizedCode.includes("invalid") + ? "invalid_request" + : normalizedCode.includes("content") || normalizedCode.includes("safety") + ? "content_policy" + : normalizedCode.includes("timeout") + ? "timeout" + : normalizedCode.includes("network") + ? "network" + : "provider_internal"; + throw new OpenRouterImageCodecError(code, safeProviderMessage(rawMessage, secretValues), { + category, + retryable: + category === "rate_limit" || + category === "overloaded" || + category === "timeout" || + category === "network", + }); +} + +async function validateWrittenBlob( + rawReference: BlobReference, + bytes: Uint8Array, + expectedMediaType: string, + index: number, +): Promise { + let reference: BlobReference; + try { + reference = parseBlobReference(rawReference, `imageResult.images[${index}]`); + } catch (error) { + responseError(error instanceof Error ? error.message : `Stored image ${index} is malformed`); + } + if ( + reference.sizeBytes !== bytes.length || + reference.sha256 !== digest(bytes) || + reference.mediaType !== expectedMediaType + ) { + responseError(`Stored image ${index} does not match the generated bytes`); + } + return reference; +} + +/** Decodes and stores one buffered OpenRouter image response. */ +export async function decodeOpenRouterImageResponse( + rawResponse: unknown, + options: OpenRouterImageDecodeOptions, +): Promise { + checkCancellation(options.request.signal); + validateModel(options.model, options.request); + validateRequest(options.request); + const response = object(rawResponse); + if (response === undefined) responseError("OpenRouter image response must be an object"); + if (response.error !== undefined) providerFailure(response, options.secretValues); + if (!Array.isArray(response.data) || response.data.length === 0) { + responseError("OpenRouter image response contains no images"); + } + if (response.data.length > MAX_OUTPUT_IMAGES) { + responseError(`OpenRouter image response exceeds ${MAX_OUTPUT_IMAGES} images`); + } + + const responseId = response.id ?? response.response_id; + if ( + responseId !== undefined && + (typeof responseId !== "string" || + responseId.length === 0 || + Buffer.byteLength(responseId) > MAX_RESPONSE_TEXT_BYTES) + ) { + responseError("OpenRouter image response ID must be a non-empty string"); + } + const routedModelId = response.model; + if ( + routedModelId !== undefined && + (typeof routedModelId !== "string" || + routedModelId.length === 0 || + Buffer.byteLength(routedModelId) > MAX_RESPONSE_TEXT_BYTES) + ) { + responseError("OpenRouter routed model must be a non-empty string"); + } + + const images: BlobReference[] = []; + const revisedPrompts = new Set(); + let decodedBytes = 0; + for (const [index, rawImage] of response.data.entries()) { + checkCancellation(options.request.signal); + const image = object(rawImage); + if (image === undefined) responseError(`OpenRouter image ${index} must be an object`); + const bytes = decodeBase64(image.b64_json, index); + decodedBytes += bytes.byteLength; + if (decodedBytes > MAX_TOTAL_DECODED_IMAGE_BYTES) + responseError("OpenRouter image response exceeds its decoded byte limit"); + const imageMediaType = mediaType(image.media_type, bytes, index); + if (image.revised_prompt !== undefined) { + if ( + typeof image.revised_prompt !== "string" || + image.revised_prompt.length === 0 || + Buffer.byteLength(image.revised_prompt) > MAX_RESPONSE_TEXT_BYTES + ) { + responseError(`OpenRouter image ${index} has an invalid revised prompt`); + } + revisedPrompts.add(image.revised_prompt); + } + const written = await options.request.writeBlob(bytes, { mediaType: imageMediaType }); + checkCancellation(options.request.signal); + images.push(await validateWrittenBlob(written, bytes, imageMediaType, index)); + } + if (revisedPrompts.size > 1) { + responseError("OpenRouter returned conflicting revised prompts"); + } + + const usage = parseUsage(response.usage, options.model); + return { + providerId: options.model.providerId, + requestedModelId: options.request.modelId, + ...(routedModelId === undefined || routedModelId === options.request.modelId + ? {} + : { routedModelId }), + ...(responseId === undefined ? {} : { responseId }), + images, + ...(revisedPrompts.size === 0 ? {} : { revisedPrompt: [...revisedPrompts][0] }), + ...(usage === undefined ? {} : { usage }), + }; +} diff --git a/packages/ai/src/provider-port.ts b/packages/ai/src/provider-port.ts index fe1a855..3025ffc 100644 --- a/packages/ai/src/provider-port.ts +++ b/packages/ai/src/provider-port.ts @@ -16,11 +16,16 @@ import { DEFAULT_MODEL_REQUEST_SETTINGS } from "@axl/protocol"; import { fitModelRequest } from "./request-configuration.ts"; import type { ModelProvider } from "./provider.ts"; +import type { RequestModelMessage } from "./model.ts"; +import type { ProviderRegistry } from "./registry.ts"; +import { prepareModelRequest } from "./request-preparation.ts"; +import { withResolvedSecretSink } from "./secret-context.ts"; import { normalizeModelStream } from "./stream.ts"; export interface SessionPortOptions { readonly modelId: string; readonly thinkingLevel?: ThinkingLevel; + readonly onResolvedSecrets?: (values: readonly string[]) => void; readonly maxOutputTokens?: number; readonly requestSettings?: ModelRequestSettings; readonly readBlob?: (reference: BlobReference) => Promise; @@ -41,64 +46,269 @@ interface PortTurnRequest { /** * Binds a provider and model choice into the shape the kernel's ModelPort - * expects (satisfied structurally — the kernel never imports this package). + * expects. It is satisfied structurally, and the kernel never imports this package. * Streams are normalized, so the kernel always sees exactly one terminal. */ +async function configureRequest( + model: Parameters[0], + request: PortTurnRequest, + options: SessionPortOptions, + messages: readonly RequestModelMessage[], +) { + const settings = options.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS; + const requestedMaximum = + request.maxOutputTokens ?? options.maxOutputTokens ?? settings.maxOutputTokens ?? undefined; + const raw = { + modelId: options.modelId, + ...(request.system === undefined ? {} : { system: request.system }), + messages, + tools: request.tools, + ...(options.thinkingLevel === undefined ? {} : { thinkingLevel: options.thinkingLevel }), + ...(requestedMaximum === undefined ? {} : { maxOutputTokens: requestedMaximum }), + httpIdleTimeoutMs: settings.httpIdleTimeoutMs, + ...(request.estimatedInputTokens === undefined + ? {} + : { estimatedInputTokens: request.estimatedInputTokens }), + ...(request.toolChoice === undefined ? {} : { toolChoice: request.toolChoice }), + ...(options.readBlob === undefined ? {} : { readBlob: options.readBlob }), + ...(request.signal === undefined ? {} : { signal: request.signal }), + }; + const configuration = fitModelRequest(model, raw); + const prepared = await prepareModelRequest(model, raw); + await request.onRequestConfigured?.(configuration); + return Object.freeze({ + ...prepared, + maxOutputTokens: configuration.maxOutputTokens, + httpIdleTimeoutMs: configuration.httpIdleTimeoutMs, + estimatedInputTokens: configuration.estimatedInputTokens, + }); +} + +type ReplayEvent = Extract; + +function replayContinuation(event: ReplayEvent) { + return { + providerId: event.providerId, + apiDialect: event.apiDialect, + modelId: event.modelId, + ...(event.responseId === undefined ? {} : { responseId: event.responseId }), + ...(event.itemId === undefined ? {} : { itemId: event.itemId }), + ...(event.namespace === undefined ? {} : { namespace: event.namespace }), + }; +} + +function hasReplayContinuation(event: ReplayEvent): boolean { + return ( + event.responseId !== undefined || event.itemId !== undefined || event.namespace !== undefined + ); +} + +function retainReplayMetadata( + messages: readonly ModelMessage[], + turns: readonly (readonly ReplayEvent[])[], +): readonly RequestModelMessage[] { + if (turns.length === 0) return messages; + const assistantIndexes = messages.flatMap((message, index) => + message.role === "assistant" ? [index] : [], + ); + const offset = Math.max(0, assistantIndexes.length - turns.length); + const byMessage = new Map(); + turns.forEach((turn, index) => { + const messageIndex = assistantIndexes[offset + index]; + if (messageIndex !== undefined) byMessage.set(messageIndex, turn); + }); + + return messages.map((message, messageIndex): RequestModelMessage => { + const replay = byMessage.get(messageIndex); + if (message.role !== "assistant" || replay === undefined || replay.length === 0) return message; + const first = replay[0]; + if (first === undefined) return message; + for (const event of replay) { + if ( + event.providerId !== first.providerId || + event.apiDialect !== first.apiDialect || + event.modelId !== first.modelId + ) { + throw new Error("A model turn emitted replay metadata for multiple model identities"); + } + } + + const thinking = replay + .filter((event) => event.target === "thinking") + .sort((a, b) => a.contentIndex - b.contentIndex); + const text = replay + .filter((event) => event.target === "text") + .sort((a, b) => a.contentIndex - b.contentIndex); + let thinkingIndex = 0; + let textIndex = 0; + const content = message.content.map((item) => { + if (item.type === "thinking") { + const event = thinking[thinkingIndex++]; + return event?.signature === undefined + ? item + : { + ...item, + signature: { + providerId: event.providerId, + apiDialect: event.apiDialect, + modelId: event.modelId, + value: event.signature, + }, + ...(event.redacted === true ? { redacted: true } : {}), + }; + } + if (item.type === "text") { + const event = text[textIndex++]; + return event === undefined + ? item + : { + ...item, + ...(event.signature === undefined + ? {} + : { + signature: { + providerId: event.providerId, + apiDialect: event.apiDialect, + modelId: event.modelId, + value: event.signature, + }, + }), + ...(hasReplayContinuation(event) ? { continuation: replayContinuation(event) } : {}), + }; + } + return item; + }); + const calls = message.toolCalls?.map((call) => { + const event = replay.find( + (candidate) => candidate.target === "tool_call" && candidate.callId === call.callId, + ); + return event === undefined + ? call + : { + ...call, + ...(event.signature === undefined + ? {} + : { + signature: { + providerId: event.providerId, + apiDialect: event.apiDialect, + modelId: event.modelId, + value: event.signature, + }, + }), + ...(hasReplayContinuation(event) ? { continuation: replayContinuation(event) } : {}), + }; + }); + const response = replay.find((event) => event.responseId !== undefined); + return { + role: "assistant", + content, + ...(calls === undefined ? {} : { toolCalls: calls }), + origin: { + providerId: first.providerId, + apiDialect: first.apiDialect, + modelId: first.modelId, + }, + ...(response === undefined ? {} : { continuation: replayContinuation(response) }), + }; + }); +} + +function streamWithSecretSink( + stream: AsyncIterable, + sink: ((values: readonly string[]) => void) | undefined, +): AsyncIterable { + return (async function* () { + const iterator = stream[Symbol.asyncIterator](); + try { + for (;;) { + const result = await withResolvedSecretSink(sink, () => iterator.next()); + if (result.done) return; + yield result.value; + } + } finally { + const close = iterator.return; + if (close !== undefined) { + await withResolvedSecretSink(sink, () => close.call(iterator)); + } + } + })(); +} + +function retainStream( + stream: AsyncIterable, + turns: ReplayEvent[][], +): AsyncIterable { + return (async function* () { + const replay: ReplayEvent[] = []; + try { + for await (const event of stream) { + if (event.type === "replay_metadata") replay.push(event); + yield event; + } + } finally { + turns.push(replay); + } + })(); +} + export function modelPortForSession( provider: ModelProvider, options: SessionPortOptions, ): { stream(request: PortTurnRequest): AsyncIterable } { + const replayTurns: ReplayEvent[][] = []; return { stream: (request) => - normalizeModelStream( - (async function* () { - if (request.signal?.aborted) { - yield { type: "aborted" } as const; - return; - } - const model = (await provider.listModels()).find( - (candidate) => candidate.modelId === options.modelId, - ); - if (model === undefined) - throw new Error(`Provider ${provider.id} has no model ${options.modelId}`); - const settings = options.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS; - const maxOutputTokens = - request.maxOutputTokens ?? - options.maxOutputTokens ?? - settings.maxOutputTokens ?? - undefined; - const configuration = fitModelRequest(model, { - messages: request.messages, - tools: request.tools, - ...(request.system === undefined ? {} : { system: request.system }), - ...(request.estimatedInputTokens === undefined - ? {} - : { estimatedInputTokens: request.estimatedInputTokens }), - ...(maxOutputTokens === undefined ? {} : { maxOutputTokens }), - httpIdleTimeoutMs: settings.httpIdleTimeoutMs, - }); - await request.onRequestConfigured?.(configuration); - if (request.signal?.aborted) { - yield { type: "aborted" } as const; - return; - } - yield* provider.stream({ - modelId: options.modelId, - ...(request.system === undefined ? {} : { system: request.system }), - messages: request.messages, - tools: request.tools, - ...(options.thinkingLevel === undefined - ? {} - : { thinkingLevel: options.thinkingLevel }), - maxOutputTokens: configuration.maxOutputTokens, - httpIdleTimeoutMs: configuration.httpIdleTimeoutMs, - estimatedInputTokens: configuration.estimatedInputTokens, - ...(request.toolChoice === undefined ? {} : { toolChoice: request.toolChoice }), - ...(options.readBlob === undefined ? {} : { readBlob: options.readBlob }), - ...(request.signal === undefined ? {} : { signal: request.signal }), - }); - })(), - request.signal, + retainStream( + normalizeModelStream( + (async function* () { + request.signal?.throwIfAborted(); + const model = (await provider.listModels()).find( + (candidate) => candidate.modelId === options.modelId, + ); + if (model === undefined) { + throw new Error(`Provider ${provider.id} has no model ${options.modelId}`); + } + const messages = retainReplayMetadata(request.messages, replayTurns); + const prepared = await configureRequest(model, request, options, messages); + request.signal?.throwIfAborted(); + yield* streamWithSecretSink(provider.stream(prepared), options.onResolvedSecrets); + })(), + request.signal, + ), + replayTurns, + ), + }; +} + +export interface RegistrySessionPortOptions extends SessionPortOptions { + readonly providerId: string; +} + +/** Binds a provider and model identity through the registry coordinator. */ +export function modelPortForRegistry( + registry: ProviderRegistry, + options: RegistrySessionPortOptions, +): { stream(request: PortTurnRequest): AsyncIterable } { + const replayTurns: ReplayEvent[][] = []; + return { + stream: (request) => + retainStream( + normalizeModelStream( + (async function* () { + request.signal?.throwIfAborted(); + const model = await registry.getModel(options.providerId, options.modelId); + const messages = retainReplayMetadata(request.messages, replayTurns); + const configured = await configureRequest(model, request, options, messages); + request.signal?.throwIfAborted(); + yield* streamWithSecretSink( + registry.stream(options.providerId, configured), + options.onResolvedSecrets, + ); + })(), + request.signal, + ), + replayTurns, ), }; } diff --git a/packages/ai/src/provider.ts b/packages/ai/src/provider.ts index 523d242..306a061 100644 --- a/packages/ai/src/provider.ts +++ b/packages/ai/src/provider.ts @@ -1,7 +1,47 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan // SPDX-License-Identifier: Apache-2.0 -import type { AuthMethod, ModelInfo, ModelRequest, ModelStreamEvent } from "./model.ts"; +import type { ProviderAuthentication } from "./auth.ts"; +import type { CatalogSnapshot, CatalogSourceMetadata } from "./catalog-store.ts"; +import type { + AuthMethod, + ImageGenerationRequest, + ImageGenerationResult, + ImageModelInfo, + ModelInfo, + ModelRequest, + ModelStreamEvent, + SafeProviderDiagnostic, +} from "./model.ts"; + +export interface ModelCatalogRefreshContext { + readonly providerId: string; + /** Provider-instance-local generation. Results from older generations are never published. */ + readonly generation: number; + /** Validated last-known-good snapshot restored before network work begins. */ + readonly previous?: Readonly; + readonly signal: AbortSignal; +} + +interface ModelCatalogRefreshMetadata { + readonly providerId: string; + readonly generation: number; + /** Safe public source metadata. URLs, headers, and credentials are intentionally absent. */ + readonly source: CatalogSourceMetadata; + readonly diagnostics?: readonly SafeProviderDiagnostic[]; +} + +export type ModelCatalogRefreshResult = + | (ModelCatalogRefreshMetadata & { + readonly status: "updated"; + readonly models: readonly ModelInfo[]; + readonly imageModels?: readonly ImageModelInfo[]; + readonly sourceUpdatedAt?: number; + readonly etag?: string; + }) + | (ModelCatalogRefreshMetadata & { + readonly status: "not_modified"; + }); /** Handle for a provider-side deferred response. Optional; no provider implements it yet. */ export interface DeferredResponse { @@ -14,16 +54,26 @@ export interface ModelProvider { readonly id: string; readonly displayName: string; readonly authMethods: readonly AuthMethod[]; + /** Provider-owned, UI-neutral authentication lifecycle when authentication is configurable. */ + readonly authentication?: ProviderAuthentication; listModels(): Promise; - /** Optional live catalog refresh; providers without it have a static catalog. */ + /** Legacy additive refresh shape retained for existing provider implementations. */ refreshModels?(): Promise; + /** Context-aware explicit live catalog refresh with persistence metadata. */ + refreshModelCatalog?(context: ModelCatalogRefreshContext): Promise; /** * Streams one model response. Failures before dispatch may throw; failures * after dispatch must terminate through a terminal stream event. Consumers * enforce this with `normalizeModelStream`. */ stream(request: ModelRequest): AsyncIterable; + /** Dispatches a registry-resolved model, including a restored dynamic catalog model. */ + streamModel?(model: ModelInfo, request: ModelRequest): AsyncIterable; /** Optional deferred-response seam. */ defer?(request: ModelRequest): Promise; + /** Optional native image catalog owned by this same provider. */ + listImageModels?(): Promise; + /** Optional native image generation. Generated bytes must use the request blob writer. */ + generateImages?(request: ImageGenerationRequest): Promise; dispose?(): void | Promise; } diff --git a/packages/ai/src/qwen-token-plan-cn.ts b/packages/ai/src/qwen-token-plan-cn.ts new file mode 100644 index 0000000..532dea6 --- /dev/null +++ b/packages/ai/src/qwen-token-plan-cn.ts @@ -0,0 +1,25 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const QWEN_TOKEN_PLAN_CN_PROVIDER_ID = "qwen-token-plan-cn"; +export const QWEN_TOKEN_PLAN_CN_API_KEY_ENV = "QWEN_TOKEN_PLAN_CN_API_KEY"; +export const QWEN_TOKEN_PLAN_CN_BASE_URL = + "https://token-plan.cn-beijing.maas.aliyuncs.com/compatible-mode/v1"; + +export const QWEN_TOKEN_PLAN_CN_PROVIDER_DEFINITION = { + id: QWEN_TOKEN_PLAN_CN_PROVIDER_ID, + displayName: "Qwen Token Plan China", + apiKeyDisplayName: "Qwen Token Plan China API key", + environmentVariables: [QWEN_TOKEN_PLAN_CN_API_KEY_ENV], + baseUrl: QWEN_TOKEN_PLAN_CN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createQwenTokenPlanCnProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(QWEN_TOKEN_PLAN_CN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/qwen-token-plan-individual.ts b/packages/ai/src/qwen-token-plan-individual.ts new file mode 100644 index 0000000..56a4818 --- /dev/null +++ b/packages/ai/src/qwen-token-plan-individual.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const QWEN_TOKEN_PLAN_INDIVIDUAL_PROVIDER_ID = "qwen-token-plan-individual"; +export const QWEN_TOKEN_PLAN_INDIVIDUAL_API_KEY_ENV = "QWEN_TOKEN_PLAN_API_KEY"; +export const QWEN_TOKEN_PLAN_INDIVIDUAL_BASE_URL = "https://coding-intl.dashscope.aliyuncs.com/v1"; + +export const QWEN_TOKEN_PLAN_INDIVIDUAL_PROVIDER_DEFINITION = { + id: QWEN_TOKEN_PLAN_INDIVIDUAL_PROVIDER_ID, + displayName: "Qwen Token Plan Individual", + apiKeyDisplayName: "Qwen Token Plan Individual API key", + environmentVariables: [QWEN_TOKEN_PLAN_INDIVIDUAL_API_KEY_ENV], + baseUrl: QWEN_TOKEN_PLAN_INDIVIDUAL_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createQwenTokenPlanIndividualProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(QWEN_TOKEN_PLAN_INDIVIDUAL_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/qwen-token-plan.ts b/packages/ai/src/qwen-token-plan.ts new file mode 100644 index 0000000..8cde76b --- /dev/null +++ b/packages/ai/src/qwen-token-plan.ts @@ -0,0 +1,25 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const QWEN_TOKEN_PLAN_PROVIDER_ID = "qwen-token-plan"; +export const QWEN_TOKEN_PLAN_API_KEY_ENV = "QWEN_TOKEN_PLAN_API_KEY"; +export const QWEN_TOKEN_PLAN_BASE_URL = + "https://token-plan.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"; + +export const QWEN_TOKEN_PLAN_PROVIDER_DEFINITION = { + id: QWEN_TOKEN_PLAN_PROVIDER_ID, + displayName: "Qwen Token Plan", + apiKeyDisplayName: "Qwen Token Plan API key", + environmentVariables: [QWEN_TOKEN_PLAN_API_KEY_ENV], + baseUrl: QWEN_TOKEN_PLAN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createQwenTokenPlanProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(QWEN_TOKEN_PLAN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/registry.ts b/packages/ai/src/registry.ts index 4c01a52..14270f0 100644 --- a/packages/ai/src/registry.ts +++ b/packages/ai/src/registry.ts @@ -1,49 +1,785 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 -import type { ModelProvider } from "./provider.ts"; +import { + type CatalogSnapshot, + type CatalogStore, + InMemoryCatalogStore, + validateCatalogSnapshot, + validateCatalogSource, +} from "./catalog-store.ts"; +import { validateModelCatalog } from "./catalog-validation.ts"; +import type { + ImageModelInfo, + ModelInfo, + ModelRequest, + ModelStreamEvent, + SafeProviderDiagnostic, +} from "./model.ts"; +import type { ModelCatalogRefreshResult, ModelProvider } from "./provider.ts"; +import { isPreparedModelRequest, prepareModelRequest } from "./request-preparation.ts"; + +export type ProviderRegistryErrorCode = + | "registry_disposed" + | "provider_duplicate" + | "provider_missing" + | "provider_disabled" + | "model_duplicate" + | "model_missing" + | "model_unavailable" + | "catalog_failure"; export class ProviderRegistryError extends Error { - constructor(message: string) { - super(message); + readonly code: ProviderRegistryErrorCode; + readonly providerId: string | undefined; + readonly modelId: string | undefined; + + constructor( + code: ProviderRegistryErrorCode, + message: string, + options: { providerId?: string; modelId?: string; cause?: unknown } = {}, + ) { + super(message, options.cause === undefined ? undefined : { cause: options.cause }); this.name = "ProviderRegistryError"; + this.code = code; + this.providerId = options.providerId; + this.modelId = options.modelId; } } +export interface ProviderRegistrationOptions { + readonly enabled?: boolean; +} + +export interface RegisteredProvider { + readonly provider: ModelProvider; + readonly enabled: boolean; +} + +export interface ModelCatalogResult { + readonly models: readonly ModelInfo[]; + /** Failures remain visible without preventing healthy providers from listing models. */ + readonly errors: ReadonlyMap; +} + +export interface ModelCatalogOptions { + readonly providerId?: string; + readonly includeUnavailable?: boolean; +} + +export interface RestoreCatalogsOptions { + readonly providerId?: string; + readonly signal?: AbortSignal; +} + +export interface RestoreCatalogsResult extends ModelCatalogResult { + readonly restoredProviderIds: readonly string[]; + readonly snapshots: ReadonlyMap; +} + +export interface RefreshProvidersOptions extends RestoreCatalogsOptions { + readonly configuredOnly?: boolean; +} + +export interface RefreshProvidersResult extends ModelCatalogResult { + readonly refreshedProviderIds: readonly string[]; + readonly restoredProviderIds: readonly string[]; + readonly supersededProviderIds: readonly string[]; + readonly snapshots: ReadonlyMap; + readonly diagnostics: ReadonlyMap; +} + +export interface ProviderRegistryOptions { + readonly catalogStore?: CatalogStore; + readonly now?: () => number; +} + +interface RegistryEntry { + readonly provider: ModelProvider; + enabled: boolean; +} + +interface ProviderRefreshState { + readonly generation: number; + readonly controller: AbortController; +} + +interface ProviderRefreshSuccess { + readonly providerId: string; + readonly refreshed: boolean; + readonly restored: boolean; + readonly superseded: boolean; + readonly models: readonly ModelInfo[]; + readonly snapshot?: CatalogSnapshot; + readonly diagnostics?: readonly SafeProviderDiagnostic[]; +} + +interface ProviderRefreshFailure { + readonly providerId: string; + readonly error: Error; +} + +type RefreshableProvider = ModelProvider & + ( + | Required> + | Required> + ); + +function available(model: ModelInfo): boolean { + return model.availability?.status !== "unavailable"; +} + +function asError(error: unknown, providerId: string, operation: string): Error { + if (error instanceof Error) return error; + return new ProviderRegistryError( + "catalog_failure", + `Provider ${providerId} ${operation} failed with a non-Error value`, + { providerId, cause: error }, + ); +} + +function raceWithSignal(operation: Promise, signal: AbortSignal): Promise { + signal.throwIfAborted(); + return new Promise((resolvePromise, reject) => { + const abort = () => reject(signal.reason); + signal.addEventListener("abort", abort, { once: true }); + operation.then( + (value) => { + signal.removeEventListener("abort", abort); + resolvePromise(value); + }, + (error: unknown) => { + signal.removeEventListener("abort", abort); + reject(error); + }, + ); + }); +} + +function validateDiagnostics( + diagnostics: readonly SafeProviderDiagnostic[] | undefined, + providerId: string, +): readonly SafeProviderDiagnostic[] | undefined { + if (diagnostics === undefined) return undefined; + if (!Array.isArray(diagnostics)) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${providerId} returned invalid catalog diagnostics`, + { providerId }, + ); + } + for (const diagnostic of diagnostics) { + if ( + typeof diagnostic !== "object" || + diagnostic === null || + Array.isArray(diagnostic) || + Object.keys(diagnostic).some((key) => !["code", "message", "severity"].includes(key)) || + typeof diagnostic.code !== "string" || + !/^[a-z0-9]+(?:[a-z0-9._-]*[a-z0-9])?$/i.test(diagnostic.code) || + diagnostic.code.length > 128 || + typeof diagnostic.message !== "string" || + diagnostic.message.length > 2_000 || + !["info", "warning", "error"].includes(diagnostic.severity) + ) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${providerId} returned invalid catalog diagnostics`, + { providerId }, + ); + } + } + return structuredClone(diagnostics); +} + /** - * Runtime provider registration for the session and, later, extensions. - * Registration returns a disposer that unregisters the provider and releases - * its resources; disposing twice is a no-op, and a disposer never removes a - * different provider registered later under the same ID. + * Owns registered provider lifecycles and coordinates model lookup, persisted + * dynamic catalogs, explicit refresh, availability, and dispatch. Registration + * itself performs no model, credential, network, or background work. */ export class ProviderRegistry { - private readonly providers = new Map(); + private readonly providers = new Map(); + private readonly disposal = new WeakMap>(); + private readonly catalogStore: CatalogStore; + private readonly now: () => number; + private readonly snapshots = new Map(); + private readonly refreshGenerations = new Map(); + private readonly refreshControllers = new Map(); + private readonly publicationChains = new Map>(); + private disposed = false; - register(provider: ModelProvider): () => Promise { + constructor(options: ProviderRegistryOptions = {}) { + this.catalogStore = options.catalogStore ?? new InMemoryCatalogStore(); + this.now = options.now ?? Date.now; + } + + register( + provider: ModelProvider, + options: ProviderRegistrationOptions = {}, + ): () => Promise { + this.assertActive(); if (this.providers.has(provider.id)) { - throw new ProviderRegistryError(`Provider ${provider.id} is already registered`); + throw new ProviderRegistryError( + "provider_duplicate", + `Provider ${provider.id} is already registered`, + { providerId: provider.id }, + ); } - this.providers.set(provider.id, provider); + const entry: RegistryEntry = { provider, enabled: options.enabled ?? true }; + this.providers.set(provider.id, entry); return async () => { - if (this.providers.get(provider.id) !== provider) return; + if (this.providers.get(provider.id) !== entry) return; + this.supersedeRefresh(provider.id); this.providers.delete(provider.id); - await provider.dispose?.(); + this.snapshots.delete(provider.id); + await this.disposeProvider(provider); }; } get(id: string): ModelProvider { - const provider = this.providers.get(id); - if (provider === undefined) { - throw new ProviderRegistryError(`Provider ${id} is not registered`); - } - return provider; + return this.entry(id).provider; } has(id: string): boolean { return this.providers.has(id); } + isEnabled(id: string): boolean { + return this.registeredEntry(id).enabled; + } + + setEnabled(id: string, enabled: boolean): void { + this.assertActive(); + const entry = this.registeredEntry(id); + entry.enabled = enabled; + if (!enabled) this.supersedeRefresh(id); + } + + /** Lists enabled providers. Disabled providers are visible through `registrations()`. */ list(): readonly ModelProvider[] { - return [...this.providers.values()]; + this.assertActive(); + return [...this.providers.values()] + .filter((entry) => entry.enabled) + .map((entry) => entry.provider); + } + + registrations(): readonly RegisteredProvider[] { + this.assertActive(); + return [...this.providers.values()].map((entry) => ({ + provider: entry.provider, + enabled: entry.enabled, + })); + } + + catalogSnapshot(providerId: string): CatalogSnapshot | undefined { + this.entry(providerId); + const snapshot = this.snapshots.get(providerId); + return snapshot === undefined ? undefined : structuredClone(snapshot); + } + + async listModels(options: ModelCatalogOptions = {}): Promise { + this.assertActive(); + const entries = + options.providerId === undefined ? this.enabledEntries() : [this.entry(options.providerId)]; + const results = await Promise.all( + entries.map(async ({ provider }) => { + try { + const providerModels = (await this.modelsFor(provider)).filter( + (model) => options.includeUnavailable || available(model), + ); + return { providerId: provider.id, models: providerModels }; + } catch (error) { + return { providerId: provider.id, error: asError(error, provider.id, "model listing") }; + } + }), + ); + const models: ModelInfo[] = []; + const errors = new Map(); + for (const result of results) { + if (result.error === undefined) models.push(...(result.models ?? [])); + else errors.set(result.providerId, result.error); + } + return { models, errors }; + } + + async listImageModels(providerId: string): Promise { + const provider = this.get(providerId); + const baseline = (await provider.listImageModels?.()) ?? []; + const snapshot = this.snapshots.get(providerId); + return structuredClone(snapshot?.imageModels ?? baseline); + } + + async getModel( + providerId: string, + modelId: string, + options: { includeUnavailable?: boolean } = {}, + ): Promise { + const provider = this.get(providerId); + let models: readonly ModelInfo[]; + try { + models = await this.modelsFor(provider); + } catch (error) { + if (error instanceof ProviderRegistryError) throw error; + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${providerId} model listing failed`, + { providerId, modelId, cause: error }, + ); + } + const model = models.find((candidate) => candidate.modelId === modelId); + if (model === undefined) { + throw new ProviderRegistryError( + "model_missing", + `Provider ${providerId} has no model ${modelId}`, + { providerId, modelId }, + ); + } + if (!options.includeUnavailable && !available(model)) { + throw new ProviderRegistryError( + "model_unavailable", + `Model ${providerId}/${modelId} is unavailable${ + model.availability?.reason ? `: ${model.availability.reason}` : "" + }`, + { providerId, modelId }, + ); + } + return model; + } + + stream(providerId: string, request: ModelRequest): AsyncIterable { + const registry = this; + return (async function* () { + const provider = registry.get(providerId); + const model = await registry.getModel(providerId, request.modelId); + const prepared = isPreparedModelRequest(request) + ? request + : await prepareModelRequest(model, request); + yield* provider.streamModel?.(model, prepared) ?? provider.stream(prepared); + })(); + } + + /** Restore last-known-good dynamic catalogs without credentials or network access. */ + async restoreCatalogs(options: RestoreCatalogsOptions = {}): Promise { + this.assertActive(); + options.signal?.throwIfAborted(); + const entries = this.refreshableEntries(options.providerId); + const results = await Promise.all( + entries.map(async ({ provider }) => { + const state = this.beginRefresh(provider.id); + const signal = + options.signal === undefined + ? state.controller.signal + : AbortSignal.any([options.signal, state.controller.signal]); + try { + const snapshot = await this.restoreProvider(provider, state.generation, signal); + return { providerId: provider.id, snapshot }; + } catch (error) { + if (options.signal?.aborted) throw error; + if (state.controller.signal.aborted) return { providerId: provider.id }; + return { providerId: provider.id, error: asError(error, provider.id, "catalog restore") }; + } finally { + this.finishRefresh(provider.id, state.controller); + } + }), + ); + const models: ModelInfo[] = []; + const errors = new Map(); + const snapshots = new Map(); + const restoredProviderIds: string[] = []; + for (const result of results) { + if (result.error !== undefined) errors.set(result.providerId, result.error); + else if (result.snapshot !== undefined) { + restoredProviderIds.push(result.providerId); + snapshots.set(result.providerId, result.snapshot); + models.push(...result.snapshot.models); + } + } + return { models, errors, restoredProviderIds, snapshots }; + } + + async refresh(options: RefreshProvidersOptions = {}): Promise { + this.assertActive(); + options.signal?.throwIfAborted(); + const entries = this.refreshableEntries(options.providerId); + const results = await Promise.all( + entries.map( + async ({ provider }): Promise => { + try { + if ( + options.configuredOnly && + provider.authentication !== undefined && + ( + await provider.authentication.check( + options.signal ? { signal: options.signal } : {}, + ) + ).phase === "logged_out" + ) { + return { + providerId: provider.id, + refreshed: false, + restored: false, + superseded: false, + models: [], + }; + } + return await this.refreshProvider(provider, options.signal); + } catch (error) { + options.signal?.throwIfAborted(); + return { + providerId: provider.id, + error: asError(error, provider.id, "authentication check"), + }; + } + }, + ), + ); + const refreshedProviderIds: string[] = []; + const restoredProviderIds: string[] = []; + const supersededProviderIds: string[] = []; + const models: ModelInfo[] = []; + const errors = new Map(); + const snapshots = new Map(); + const diagnostics = new Map(); + for (const result of results) { + if ("error" in result) { + errors.set(result.providerId, result.error); + continue; + } + if (result.refreshed) refreshedProviderIds.push(result.providerId); + if (result.restored) restoredProviderIds.push(result.providerId); + if (result.superseded) supersededProviderIds.push(result.providerId); + models.push(...result.models); + if (result.snapshot !== undefined) snapshots.set(result.providerId, result.snapshot); + if (result.diagnostics !== undefined) diagnostics.set(result.providerId, result.diagnostics); + } + return { + models, + errors, + refreshedProviderIds, + restoredProviderIds, + supersededProviderIds, + snapshots, + diagnostics, + }; + } + + async dispose(): Promise { + if (this.disposed) return; + this.disposed = true; + for (const id of this.refreshControllers.keys()) this.supersedeRefresh(id); + const providers = [...this.providers.values()].map((entry) => entry.provider); + this.providers.clear(); + this.snapshots.clear(); + const results = await Promise.allSettled( + providers.map((provider) => this.disposeProvider(provider)), + ); + const errors = results.flatMap((result) => + result.status === "rejected" ? [result.reason] : [], + ); + if (errors.length > 0) throw new AggregateError(errors, "Provider registry disposal failed"); + } + + private async refreshProvider( + provider: RefreshableProvider, + callerSignal: AbortSignal | undefined, + ): Promise { + const state = this.beginRefresh(provider.id); + const signal = + callerSignal === undefined + ? state.controller.signal + : AbortSignal.any([callerSignal, state.controller.signal]); + let restored = false; + try { + const previous = await this.restoreProvider(provider, state.generation, signal); + restored = previous !== undefined; + signal.throwIfAborted(); + const operation = + provider.refreshModelCatalog !== undefined + ? provider.refreshModelCatalog({ + providerId: provider.id, + generation: state.generation, + ...(previous === undefined ? {} : { previous: structuredClone(previous) }), + signal, + }) + : (provider.refreshModels as () => Promise)().then( + (models): ModelCatalogRefreshResult => ({ + status: "updated", + providerId: provider.id, + generation: state.generation, + source: { id: `${provider.id}-legacy`, kind: "provider_api" }, + models, + }), + ); + const result = await raceWithSignal(operation, signal); + signal.throwIfAborted(); + const diagnostics = validateDiagnostics(result.diagnostics, provider.id); + const snapshot = await this.snapshotFromResult( + provider, + state.generation, + previous, + result, + signal, + ); + if (snapshot === undefined) { + return { + providerId: provider.id, + refreshed: false, + restored, + superseded: true, + models: [], + }; + } + return { + providerId: provider.id, + refreshed: true, + restored, + superseded: false, + models: snapshot.models, + snapshot, + ...(diagnostics === undefined ? {} : { diagnostics }), + }; + } catch (error) { + if (callerSignal?.aborted) throw error; + if (state.controller.signal.aborted) { + return { + providerId: provider.id, + refreshed: false, + restored, + superseded: true, + models: [], + }; + } + return { providerId: provider.id, error: asError(error, provider.id, "model refresh") }; + } finally { + this.finishRefresh(provider.id, state.controller); + } + } + + private async snapshotFromResult( + provider: ModelProvider, + refreshGeneration: number, + previous: CatalogSnapshot | undefined, + result: ModelCatalogRefreshResult, + signal: AbortSignal, + ): Promise { + if (result.providerId !== provider.id || result.generation !== refreshGeneration) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${provider.id} returned mismatched catalog refresh metadata`, + { providerId: provider.id }, + ); + } + validateCatalogSource(result.source, provider.id); + const checkedAt = this.now(); + let candidate: CatalogSnapshot; + if (result.status === "not_modified") { + if (previous === undefined) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${provider.id} returned not_modified without a previous catalog`, + { providerId: provider.id }, + ); + } + candidate = { + ...previous, + checkedAt, + source: structuredClone(result.source), + }; + } else { + const models = this.validateModels(provider, result.models); + candidate = { + version: 1, + providerId: provider.id, + generation: + Math.max(previous?.generation ?? 0, this.snapshots.get(provider.id)?.generation ?? 0) + 1, + checkedAt, + updatedAt: checkedAt, + ...(result.sourceUpdatedAt === undefined + ? {} + : { sourceUpdatedAt: result.sourceUpdatedAt }), + ...(result.etag === undefined ? {} : { etag: result.etag }), + source: structuredClone(result.source), + models: structuredClone(models), + ...(result.imageModels === undefined + ? {} + : { imageModels: structuredClone(result.imageModels) }), + }; + } + const validated = validateCatalogSnapshot(candidate, provider.id); + return (await this.publishSnapshot(provider.id, refreshGeneration, validated, signal, true)) + ? validated + : undefined; + } + + private async restoreProvider( + provider: ModelProvider, + generation: number, + signal: AbortSignal, + ): Promise { + const stored = await this.catalogStore.read(provider.id, { signal }); + signal.throwIfAborted(); + if (stored === undefined) return this.snapshots.get(provider.id); + const validated = validateCatalogSnapshot(stored, provider.id); + const current = this.snapshots.get(provider.id); + const selected = + current !== undefined && current.generation > validated.generation ? current : validated; + const published = await this.publishSnapshot(provider.id, generation, selected, signal, false); + return published ? structuredClone(selected) : undefined; + } + + private publishSnapshot( + providerId: string, + generation: number, + snapshot: CatalogSnapshot, + signal: AbortSignal, + persist: boolean, + ): Promise { + const previous = this.publicationChains.get(providerId) ?? Promise.resolve(); + const queued = (async () => { + await previous.catch(() => undefined); + if (signal.aborted || this.refreshGenerations.get(providerId) !== generation) return false; + const previousSnapshot = this.snapshots.get(providerId); + if (persist) await this.catalogStore.write(providerId, snapshot, { signal }); + if (signal.aborted || this.refreshGenerations.get(providerId) !== generation) { + // A store implementation may complete its atomic replacement at the same + // instant this generation is superseded. Roll that rejected generation + // back while the provider publication queue still excludes newer writes. + if (persist) { + if (previousSnapshot === undefined) await this.catalogStore.delete(providerId); + else await this.catalogStore.write(providerId, previousSnapshot); + } + return false; + } + this.snapshots.set(providerId, structuredClone(snapshot)); + return true; + })(); + const tail = queued.catch(() => undefined); + this.publicationChains.set(providerId, tail); + void tail.then(() => { + if (this.publicationChains.get(providerId) === tail) + this.publicationChains.delete(providerId); + }); + return raceWithSignal(queued, signal); + } + + private beginRefresh(providerId: string): ProviderRefreshState { + const generation = this.supersedeRefresh(providerId); + const controller = new AbortController(); + this.refreshControllers.set(providerId, controller); + return { generation, controller }; + } + + private supersedeRefresh(providerId: string): number { + const generation = (this.refreshGenerations.get(providerId) ?? 0) + 1; + this.refreshGenerations.set(providerId, generation); + const controller = this.refreshControllers.get(providerId); + if (controller !== undefined) { + this.refreshControllers.delete(providerId); + controller.abort(new DOMException("Catalog refresh superseded", "AbortError")); + } + return generation; + } + + private finishRefresh(providerId: string, controller: AbortController): void { + if (this.refreshControllers.get(providerId) === controller) { + this.refreshControllers.delete(providerId); + } + } + + private assertActive(): void { + if (this.disposed) { + throw new ProviderRegistryError("registry_disposed", "Provider registry is disposed"); + } + } + + private registeredEntry(id: string): RegistryEntry { + const entry = this.providers.get(id); + if (entry === undefined) { + throw new ProviderRegistryError("provider_missing", `Provider ${id} is not registered`, { + providerId: id, + }); + } + return entry; + } + + private entry(id: string): RegistryEntry { + this.assertActive(); + const entry = this.registeredEntry(id); + if (!entry.enabled) { + throw new ProviderRegistryError("provider_disabled", `Provider ${id} is disabled`, { + providerId: id, + }); + } + return entry; + } + + private enabledEntries(): readonly RegistryEntry[] { + return [...this.providers.values()].filter((entry) => entry.enabled); + } + + private refreshableEntries( + providerId: string | undefined, + ): readonly (RegistryEntry & { provider: RefreshableProvider })[] { + const entries = providerId === undefined ? this.enabledEntries() : [this.entry(providerId)]; + return entries.filter( + (entry): entry is RegistryEntry & { provider: RefreshableProvider } => + entry.provider.refreshModelCatalog !== undefined || + entry.provider.refreshModels !== undefined, + ); + } + + private async modelsFor(provider: ModelProvider): Promise { + const baseline = this.validateModels(provider, await provider.listModels()); + const snapshot = this.snapshots.get(provider.id); + if (snapshot === undefined) return baseline; + const merged = [...baseline]; + for (const model of snapshot.models) { + const index = merged.findIndex((candidate) => candidate.modelId === model.modelId); + if (index < 0) merged.push(model); + else merged[index] = model; + } + return this.validateModels(provider, merged); + } + + private validateModels( + provider: ModelProvider, + models: readonly ModelInfo[], + ): readonly ModelInfo[] { + const ids = new Set(); + for (const model of models) { + if (model.providerId !== provider.id) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${provider.id} returned model ${model.modelId} owned by ${model.providerId}`, + { providerId: provider.id, modelId: model.modelId }, + ); + } + if (ids.has(model.modelId)) { + throw new ProviderRegistryError( + "model_duplicate", + `Provider ${provider.id} returned duplicate model ${model.modelId}`, + { providerId: provider.id, modelId: model.modelId }, + ); + } + ids.add(model.modelId); + } + try { + validateModelCatalog(models); + } catch (error) { + throw new ProviderRegistryError( + "catalog_failure", + `Provider ${provider.id} returned an invalid model catalog`, + { providerId: provider.id, cause: error }, + ); + } + return models; + } + + private disposeProvider(provider: ModelProvider): Promise { + const existing = this.disposal.get(provider); + if (existing !== undefined) return existing; + const operation = Promise.resolve().then(() => provider.dispose?.()); + this.disposal.set(provider, operation); + return operation; } } diff --git a/packages/ai/src/remaining-providers.ts b/packages/ai/src/remaining-providers.ts new file mode 100644 index 0000000..e59fca0 --- /dev/null +++ b/packages/ai/src/remaining-providers.ts @@ -0,0 +1,1338 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + decodeAnthropicMessagesStream, + encodeAnthropicMessagesRequest, +} from "./anthropic-messages.ts"; +import { createEnvironmentApiKeyAuth } from "./api-key-auth.ts"; +import { + type AmbientAuthSource, + type ApiKeyAuthMethod, + type AuthContext, + AuthError, + createProviderAuthentication, + type ResolvedAuth, +} from "./auth.ts"; +import { + type AwsAuthFactories, + createBedrockSources, + createBedrockStoredAuth, +} from "./aws-auth.ts"; +import { decodeAwsEventStream } from "./aws-event-stream.ts"; +import { encodeAzureOpenAiResponsesRequest } from "./azure-openai.ts"; +import { + decodeBedrockConverseStream, + encodeBedrockConverseStreamRequest, +} from "./bedrock-converse-stream.ts"; +import { getStaticModelCatalog } from "./catalog.ts"; +import { validateModelCatalog } from "./catalog-validation.ts"; +import { + type CloudAuthFactories, + createAzureEntraSource, + createGoogleVertexSources, + createGoogleVertexStoredAuth, + vertexRequestPolicy, +} from "./cloud-auth.ts"; +import type { CredentialStore } from "./credentials.ts"; +import { decodeGatewayMessagesStream, encodeGatewayMessagesRequest } from "./gateway-messages.ts"; +import { + decodeGoogleGenerativeAiStream, + encodeGoogleGenerativeAiRequest, +} from "./google-generative-ai.ts"; +import { decodeGoogleVertexStream, encodeGoogleVertexRequest } from "./google-vertex.ts"; +import { type HttpSseCodec, HttpSseProvider } from "./http-sse-provider.ts"; +import { + decodeMistralConversationsStream, + encodeMistralConversationsRequest, +} from "./mistral-conversations.ts"; +import type { ImageGenerationRequest, ImageModelInfo, ModelInfo } from "./model.ts"; +import { + createAnthropicOAuth, + createGitHubCopilotOAuth, + createGitHubCopilotTokenAuth, + createKimiCodingOAuth, + createOpenAiCodexOAuth, + createOpenRouterOAuth, + createRadiusOAuth, +} from "./oauth-auth.ts"; +import { decodeOpenAiChatStream, encodeOpenAiChatRequest } from "./openai-chat.ts"; +import { + decodeOpenAiCodexResponsesStream, + encodeOpenAiCodexResponsesRequest, +} from "./openai-codex-responses.ts"; +import { decodeResponsesStream, encodeResponsesRequest } from "./openai-responses.ts"; +import { + decodeOpenRouterImageResponse, + encodeOpenRouterImageRequest, +} from "./openrouter-images.ts"; +import type { ModelCatalogRefreshContext, ModelProvider } from "./provider.ts"; +import { createStaticOpenAiChatProvider } from "./static-openai-chat-provider.ts"; +import { + delayWithSignal, + raceWithSignal, + readBoundedJson, + safeEndpoint, + safeFetch, + stripTrailingSlashes, +} from "./transport-safety.ts"; + +export interface ProviderFactoryOptions { + readonly store: CredentialStore; + readonly context: AuthContext; + readonly fetch?: typeof fetch; + readonly now?: () => number; + readonly cloudAuth?: CloudAuthFactories; + readonly awsAuth?: AwsAuthFactories; +} + +const fixedBase = (model: ModelInfo): string => { + if (model.endpoint?.type !== "fixed") + throw new TypeError(`Model ${model.modelId} has no fixed endpoint`); + return stripTrailingSlashes(model.endpoint.baseUrl); +}; + +function bearer(resolved: ResolvedAuth, providerId: string): string { + const key = resolved.auth.apiKey; + if (!key) + throw new AuthError( + "invalid_auth", + providerId, + `Provider ${providerId} has no resolved API key`, + ); + return `Bearer ${key}`; +} + +function codecs( + providerId: string, + options: { anthropicBearer?: boolean; keyless?: boolean } = {}, +): (model: ModelInfo) => HttpSseCodec { + const authorization = (resolved: ResolvedAuth): Readonly> => + options.keyless === true && !resolved.auth.apiKey + ? { ...resolved.auth.headers } + : { ...resolved.auth.headers, authorization: bearer(resolved, providerId) }; + const base = (model: ModelInfo, resolved: ResolvedAuth): string => + resolved.auth.baseUrl === undefined + ? fixedBase(model) + : stripTrailingSlashes(resolved.auth.baseUrl); + return (model) => { + if (model.apiDialect === "openai-chat") { + return { + encode: (selected, request, resolved) => ({ + url: `${base(selected, resolved)}/chat/completions`, + headers: { ...selected.headers, ...authorization(resolved) }, + body: encodeOpenAiChatRequest(selected, request).body, + }), + decode: (frames, decodeOptions) => decodeOpenAiChatStream(frames, decodeOptions), + }; + } + if (model.apiDialect === "openai-responses") { + return { + encode: (selected, request, resolved) => ({ + url: `${base(selected, resolved)}/responses`, + headers: { ...selected.headers, ...authorization(resolved) }, + body: encodeResponsesRequest(selected, request).body, + }), + decode: (frames, decodeOptions) => decodeResponsesStream(frames, decodeOptions), + }; + } + if (model.apiDialect === "anthropic-messages") { + return { + encode: (selected, request, resolved) => { + const encoded = encodeAnthropicMessagesRequest(selected, request); + const authorization = + resolved.auth.headers?.authorization ?? bearer(resolved, providerId); + const endpoint = base(selected, resolved); + return { + url: `${endpoint}${endpoint.endsWith("/v1") ? "" : "/v1"}/messages`, + headers: { + ...selected.headers, + ...encoded.headers, + ...resolved.auth.headers, + ...(options.anthropicBearer || resolved.auth.headers?.authorization !== undefined + ? { authorization } + : { "x-api-key": resolved.auth.apiKey ?? "" }), + ...(encoded.headers["anthropic-beta"] !== undefined && + resolved.auth.headers?.["anthropic-beta"] !== undefined + ? { + "anthropic-beta": `${encoded.headers["anthropic-beta"]},${resolved.auth.headers["anthropic-beta"]}`, + } + : {}), + }, + body: encoded.body, + }; + }, + decode: (frames, decodeOptions) => decodeAnthropicMessagesStream(frames, decodeOptions), + }; + } + if (model.apiDialect === "google-generative-ai") { + return { + encode: (selected, request, resolved) => { + const encoded = encodeGoogleGenerativeAiRequest(selected, request); + return { + url: `${base(selected, resolved)}/models/${encodeURIComponent(selected.modelId)}:streamGenerateContent?alt=sse`, + headers: { + ...selected.headers, + ...encoded.headers, + "x-goog-api-key": resolved.auth.apiKey ?? "", + }, + body: encoded.body, + }; + }, + decode: (frames, decodeOptions) => decodeGoogleGenerativeAiStream(frames, decodeOptions), + }; + } + if (model.apiDialect === "mistral-conversations") { + return { + encode: (selected, request, resolved) => { + const encoded = encodeMistralConversationsRequest(selected, request); + return { + url: `${base(selected, resolved)}/conversations`, + headers: { ...selected.headers, ...encoded.headers, ...authorization(resolved) }, + body: encoded.body, + }; + }, + decode: (frames, decodeOptions) => decodeMistralConversationsStream(frames, decodeOptions), + }; + } + if (model.apiDialect === "gateway-messages") { + return { + encode: (selected, request, resolved) => ({ + url: `${base(selected, resolved)}/messages`, + headers: { ...selected.headers, ...authorization(resolved) }, + body: encodeGatewayMessagesRequest(selected, request).body, + }), + decode: (frames, decodeOptions) => decodeGatewayMessagesStream(frames, decodeOptions), + }; + } + throw new TypeError(`Provider ${providerId} has no transport for ${model.apiDialect}`); + }; +} + +function apiKeyProvider(input: { + allowLoopbackHttp?: boolean; + id: string; + displayName: string; + environmentVariables: readonly string[]; + options: ProviderFactoryOptions; + models?: readonly ModelInfo[]; + codecFor?: (model: ModelInfo) => HttpSseCodec; + oauth?: ReturnType; + validateEndpoint?: (url: URL, model: ModelInfo, resolved: ResolvedAuth) => void; +}): HttpSseProvider { + const method = createEnvironmentApiKeyAuth({ + providerId: input.id, + displayName: `${input.displayName} API key`, + environmentVariables: input.environmentVariables, + }); + const authentication = createProviderAuthentication({ + providerId: input.id, + declaredMethods: + input.oauth === undefined ? ["environment", "file"] : ["environment", "file", "oauth"], + methods: { apiKey: method, ...(input.oauth === undefined ? {} : { oauth: input.oauth }) }, + store: input.options.store, + context: input.options.context, + }); + return new HttpSseProvider({ + id: input.id, + ...(input.allowLoopbackHttp === undefined + ? {} + : { allowLoopbackHttp: input.allowLoopbackHttp }), + displayName: input.displayName, + authMethods: authentication.methods, + authentication, + models: input.models ?? getStaticModelCatalog(input.id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: input.codecFor ?? codecs(input.id), + ...(input.validateEndpoint === undefined ? {} : { validateEndpoint: input.validateEndpoint }), + ...(input.options.fetch === undefined ? {} : { fetch: input.options.fetch }), + ...(input.options.now === undefined ? {} : { now: input.options.now }), + }); +} + +export const createOpenAiProvider = (options: ProviderFactoryOptions): ModelProvider => + apiKeyProvider({ + id: "openai", + displayName: "OpenAI", + environmentVariables: ["OPENAI_API_KEY"], + options, + }); + +export const createAnthropicProvider = (options: ProviderFactoryOptions): ModelProvider => { + const id = "anthropic"; + const apiKey = createEnvironmentApiKeyAuth({ + providerId: id, + displayName: "Anthropic API key", + environmentVariables: ["ANTHROPIC_API_KEY"], + }); + const oauthEnvironment: AmbientAuthSource = { + type: "environment", + displayName: "Anthropic OAuth token", + resolve: async ({ context, signal }) => { + signal.throwIfAborted(); + const token = context.env("ANTHROPIC_OAUTH_TOKEN"); + if (!token) return undefined; + return { + auth: { + headers: { + authorization: `Bearer ${token}`, + "anthropic-beta": "claude-code-20250219,oauth-2025-04-20", + }, + }, + source: "ANTHROPIC_OAUTH_TOKEN", + secretValues: [token, "claude-code-20250219,oauth-2025-04-20"], + }; + }, + }; + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file", "oauth"], + methods: { + apiKey, + oauth: createAnthropicOAuth(options), + sources: [{ ...apiKey, type: "environment" }, oauthEnvironment], + }, + store: options.store, + context: options.context, + }); + return new HttpSseProvider({ + id, + displayName: "Anthropic", + authMethods: authentication.methods, + authentication, + models: getStaticModelCatalog(id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: codecs(id), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +}; + +export const createGoogleProvider = (options: ProviderFactoryOptions): ModelProvider => + apiKeyProvider({ + id: "google", + displayName: "Google Generative AI", + environmentVariables: ["GEMINI_API_KEY", "GOOGLE_API_KEY"], + options, + }); + +export const createMistralProvider = (options: ProviderFactoryOptions): ModelProvider => + apiKeyProvider({ + id: "mistral", + displayName: "Mistral", + environmentVariables: ["MISTRAL_API_KEY"], + options, + }); + +export const createKimiCodingProvider = (options: ProviderFactoryOptions): ModelProvider => + createStaticOpenAiChatProvider( + { + id: "kimi-coding", + displayName: "Kimi For Coding", + apiKeyDisplayName: "Kimi API key", + environmentVariables: ["KIMI_API_KEY"], + baseUrl: "https://api.kimi.com/coding/v1", + oauth: createKimiCodingOAuth(options), + }, + options, + ); + +export const createOpenCodeProvider = (options: ProviderFactoryOptions): ModelProvider => + apiKeyProvider({ + id: "opencode", + displayName: "OpenCode Zen", + environmentVariables: ["OPENCODE_API_KEY"], + options, + codecFor: codecs("opencode", { anthropicBearer: true }), + }); + +export const createOpenCodeGoProvider = (options: ProviderFactoryOptions): ModelProvider => + apiKeyProvider({ + id: "opencode-go", + displayName: "OpenCode Go", + environmentVariables: ["OPENCODE_API_KEY"], + options, + codecFor: codecs("opencode-go", { anthropicBearer: true }), + }); + +export function createOpenAiCodexProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "openai-codex"; + const oauth = createOpenAiCodexOAuth(options); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["oauth"], + methods: { oauth }, + store: options.store, + context: options.context, + }); + return new HttpSseProvider({ + id, + displayName: "OpenAI Codex", + authMethods: authentication.methods, + authentication, + models: getStaticModelCatalog(id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: () => ({ + encode: (model, request, resolved) => + encodeOpenAiCodexResponsesRequest(model, request, resolved), + decode: (frames, decodeOptions) => decodeOpenAiCodexResponsesStream(frames, decodeOptions), + }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} + +export function createAmazonBedrockProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "amazon-bedrock"; + const method = createBedrockStoredAuth(options.awsAuth); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file", "ambient"], + methods: { apiKey: method, sources: createBedrockSources(options.awsAuth) }, + store: options.store, + context: options.context, + }); + return new HttpSseProvider({ + id, + displayName: "Amazon Bedrock", + authMethods: authentication.methods, + authentication, + models: getStaticModelCatalog(id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: () => ({ + encode: (model, request, resolved) => { + const region = resolved.env?.AWS_REGION; + if (region === undefined) + throw new AuthError("not_configured", id, "Amazon Bedrock region is missing"); + return encodeBedrockConverseStreamRequest(model, request, { + region, + authentication: + resolved.auth.signRequest === undefined + ? { type: "bearer", token: resolved.auth.apiKey ?? "" } + : { type: "sigv4" }, + }); + }, + decode: () => { + throw new TypeError("Bedrock requires AWS event stream framing"); + }, + decodeBody: (body, decodeOptions) => + decodeBedrockConverseStream(decodeAwsEventStream(body), decodeOptions), + }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} + +const azureAuth = (providerId: string): ApiKeyAuthMethod => ({ + displayName: "Azure OpenAI API key", + login: async (interaction) => { + const key = await interaction.prompt({ + type: "secret", + message: "Enter Azure OpenAI API key", + }); + const baseUrl = await interaction.prompt({ + type: "text", + message: "Enter Azure OpenAI base URL", + }); + if (key.length === 0) throw new TypeError("Azure OpenAI API key cannot be empty"); + safeEndpoint(baseUrl, { label: "Azure OpenAI base URL", allowLoopbackHttp: true }); + return { type: "api_key", key, env: { AZURE_OPENAI_BASE_URL: baseUrl } }; + }, + resolve: async ({ context, credential, signal }) => { + signal.throwIfAborted(); + const key = credential?.key ?? context.env("AZURE_OPENAI_API_KEY"); + const baseUrl = credential?.env?.AZURE_OPENAI_BASE_URL ?? context.env("AZURE_OPENAI_BASE_URL"); + const resource = + credential?.env?.AZURE_OPENAI_RESOURCE_NAME ?? context.env("AZURE_OPENAI_RESOURCE_NAME"); + if (!key) return undefined; + if (!baseUrl && !resource) + throw new AuthError( + "not_configured", + providerId, + "Azure OpenAI requires a base URL or resource name", + ); + return { + auth: { apiKey: key }, + env: { + AZURE_OPENAI_BASE_URL: baseUrl ?? `https://${resource}.openai.azure.com/openai/v1`, + ...(context.env("AZURE_OPENAI_API_VERSION") + ? { AZURE_OPENAI_API_VERSION: context.env("AZURE_OPENAI_API_VERSION") as string } + : {}), + ...(context.env("AZURE_OPENAI_DEPLOYMENT_NAME_MAP") + ? { + AZURE_OPENAI_DEPLOYMENT_NAME_MAP: context.env( + "AZURE_OPENAI_DEPLOYMENT_NAME_MAP", + ) as string, + } + : {}), + }, + source: credential?.key ? "stored credential" : "AZURE_OPENAI_API_KEY", + secretValues: [key], + }; + }, +}); + +export function createAzureOpenAiResponsesProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "azure-openai-responses"; + const apiKey = azureAuth(id); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file", "ambient"], + methods: { + apiKey, + sources: [{ ...apiKey, type: "environment" }, createAzureEntraSource(options.cloudAuth)], + }, + store: options.store, + context: options.context, + }); + return new HttpSseProvider({ + id, + displayName: "Azure OpenAI Responses", + authMethods: authentication.methods, + authentication, + models: getStaticModelCatalog(id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: () => ({ + encode: (model, request, resolved) => + encodeAzureOpenAiResponsesRequest(model, request, resolved), + decode: (frames, decodeOptions) => decodeResponsesStream(frames, decodeOptions), + }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} + +export function createGoogleVertexProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "google-vertex"; + const apiKey = createGoogleVertexStoredAuth(options.cloudAuth); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file", "ambient"], + methods: { apiKey, sources: createGoogleVertexSources(options.cloudAuth) }, + store: options.store, + context: options.context, + }); + return new HttpSseProvider({ + id, + displayName: "Google Vertex AI", + authMethods: authentication.methods, + authentication, + models: getStaticModelCatalog(id), + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: () => ({ + encode: (model, request, resolved) => + encodeGoogleVertexRequest(model, request, vertexRequestPolicy(resolved)), + decode: (frames, decodeOptions) => decodeGoogleVertexStream(frames, decodeOptions), + }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} + +function configuredApiKey(input: { + providerId: string; + displayName: string; + keyEnv: string; + settings: readonly { env: string; key: string }[]; +}): ApiKeyAuthMethod { + return { + displayName: input.displayName, + login: async (interaction) => { + const key = await interaction.prompt({ + type: "secret", + message: `Enter ${input.displayName}`, + }); + const env: Record = {}; + for (const setting of input.settings) + env[setting.env] = await interaction.prompt({ + type: "text", + message: `Enter ${setting.key}`, + }); + return { type: "api_key", key, env }; + }, + resolve: async ({ context, credential, signal }) => { + signal.throwIfAborted(); + const key = credential?.key ?? context.env(input.keyEnv); + if (!key) return undefined; + const env: Record = {}; + for (const setting of input.settings) { + const value = credential?.env?.[setting.env] ?? context.env(setting.env); + if (!value) + throw new AuthError( + "not_configured", + input.providerId, + `${input.providerId} requires ${setting.env}`, + ); + env[setting.env] = value; + } + return { + auth: { apiKey: key }, + env, + source: credential?.key ? "stored credential" : input.keyEnv, + secretValues: [key], + }; + }, + }; +} + +export function createCloudflareWorkersAiProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "cloudflare-workers-ai"; + const method = configuredApiKey({ + providerId: id, + displayName: "Cloudflare API token", + keyEnv: "CLOUDFLARE_API_KEY", + settings: [{ env: "CLOUDFLARE_ACCOUNT_ID", key: "Cloudflare account ID" }], + }); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file"], + methods: { apiKey: method }, + store: options.store, + context: options.context, + }); + const models = getStaticModelCatalog(id); + return new HttpSseProvider({ + id, + displayName: "Cloudflare Workers AI", + authMethods: authentication.methods, + authentication, + models, + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: () => ({ + encode: (model, request, resolved) => ({ + url: `https://api.cloudflare.com/client/v4/accounts/${encodeURIComponent(resolved.env?.CLOUDFLARE_ACCOUNT_ID ?? "")}/ai/v1/chat/completions`, + headers: { authorization: bearer(resolved, id) }, + body: encodeOpenAiChatRequest(model, request).body, + }), + decode: (frames, decodeOptions) => decodeOpenAiChatStream(frames, decodeOptions), + }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + }); +} + +const MAX_DYNAMIC_MODELS = 10_000; +const MAX_DYNAMIC_NAME_LENGTH = 512; + +function dynamicModel( + providerId: string, + endpoint: string, + value: unknown, + headers?: Readonly>, +): ModelInfo { + if (typeof value !== "object" || value === null || Array.isArray(value)) + throw new TypeError(`${providerId} returned a malformed model`); + const row = value as Record; + if ( + typeof row.id !== "string" || + row.id.length === 0 || + row.id.length > 256 || + typeof row.name !== "string" || + row.name.length === 0 || + row.name.length > MAX_DYNAMIC_NAME_LENGTH + ) { + throw new TypeError(`${providerId} returned a model without a bounded identity`); + } + const rawDialect = row.apiDialect ?? row.api; + if ( + typeof rawDialect !== "string" || + ![ + "openai-chat", + "openai-responses", + "anthropic-messages", + "google-generative-ai", + "gateway-messages", + ].includes(rawDialect) + ) { + throw new TypeError(`${providerId} returned missing or unsupported dialect metadata`); + } + const dialect = rawDialect as + | "openai-chat" + | "openai-responses" + | "anthropic-messages" + | "google-generative-ai" + | "gateway-messages"; + const topProvider = + typeof row.top_provider === "object" && + row.top_provider !== null && + !Array.isArray(row.top_provider) + ? (row.top_provider as Record) + : undefined; + const contextValue = row.context_length ?? row.contextWindow; + const outputValue = topProvider?.max_completion_tokens ?? row.maxOutputTokens; + const context = Number(contextValue); + const output = Number(outputValue); + if ( + contextValue === undefined || + outputValue === undefined || + !Number.isSafeInteger(context) || + context <= 0 || + !Number.isSafeInteger(output) || + output <= 0 || + output > context + ) { + throw new TypeError(`${providerId} returned missing or invalid model limits`); + } + const architecture = + typeof row.architecture === "object" && + row.architecture !== null && + !Array.isArray(row.architecture) + ? (row.architecture as Record) + : undefined; + const input = architecture?.input_modalities; + const supported = row.supported_parameters; + if ( + !Array.isArray(input) || + input.length > 32 || + !input.every((item) => typeof item === "string" && item.length <= 128) + ) { + throw new TypeError(`${providerId} returned missing input capability metadata`); + } + if ( + !Array.isArray(supported) || + supported.length > 128 || + !supported.every((item) => typeof item === "string" && item.length <= 128) + ) { + throw new TypeError(`${providerId} returned missing supported-parameter metadata`); + } + const baseCompatibility = + dialect === "openai-chat" + ? { dialect, supportsUsageInStreaming: true, maxTokensField: "max_tokens" as const } + : { dialect }; + const model = { + providerId, + modelId: row.id, + displayName: row.name, + apiDialect: dialect, + capabilities: { + toolUse: supported.includes("tools"), + structuredOutput: supported.includes("structured_outputs"), + imageInput: input.includes("image"), + }, + reasoning: supported.includes("reasoning"), + contextWindow: context, + maxOutputTokens: output, + endpoint: { type: "fixed", baseUrl: endpoint } as const, + ...(headers === undefined ? {} : { headers }), + availability: { status: "available" as const }, + compatibility: baseCompatibility, + } satisfies ModelInfo; + validateModelCatalog([model]); + return model; +} + +function dynamicProvider(input: { + id: string; + displayName: string; + environmentVariables: readonly string[]; + baseUrl: string; + sourceKind: "provider_api" | "entitlement" | "gateway"; + options: ProviderFactoryOptions; + headers?: (resolved: ResolvedAuth) => Readonly>; + endpoint?: (resolved: ResolvedAuth) => string; + allowEndpoint?: (url: URL) => boolean; + rowFilter?: (row: unknown) => boolean; + onRows?: (rows: readonly unknown[], endpoint: string) => readonly ImageModelInfo[] | undefined; + modelHeaders?: Readonly>; + defaultDialect?: "openai-chat"; + oauth?: ReturnType; + apiKey?: ApiKeyAuthMethod; +}): ModelProvider { + const method = + input.apiKey ?? + createEnvironmentApiKeyAuth({ + providerId: input.id, + displayName: `${input.displayName} token`, + environmentVariables: input.environmentVariables, + }); + const authentication = createProviderAuthentication({ + providerId: input.id, + declaredMethods: ["environment", "file", "oauth"], + methods: { apiKey: method, ...(input.oauth === undefined ? {} : { oauth: input.oauth }) }, + store: input.options.store, + context: input.options.context, + }); + const approvedBase = (resolved: ResolvedAuth): string => { + const base = safeEndpoint(input.endpoint?.(resolved) ?? input.baseUrl, { + label: `${input.displayName} endpoint`, + }); + if ( + input.allowEndpoint !== undefined + ? !input.allowEndpoint(new URL(base)) + : new URL(base).origin !== new URL(input.baseUrl).origin + ) { + throw new TypeError(`${input.displayName} endpoint has an unapproved origin`); + } + return base; + }; + const provider = new HttpSseProvider({ + id: input.id, + displayName: input.displayName, + authMethods: authentication.methods, + authentication, + models: [], + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: codecs(input.id, { anthropicBearer: true }), + validateEndpoint: (url, _model, resolved) => { + if (url.origin !== new URL(approvedBase(resolved)).origin) + throw new TypeError(`${input.displayName} request endpoint has an unapproved origin`); + }, + ...(input.options.fetch === undefined ? {} : { fetch: input.options.fetch }), + }); + const fetchImpl = input.options.fetch; + return Object.assign(provider, { + refreshModelCatalog: async (context: ModelCatalogRefreshContext) => { + const resolved = await authentication.resolve({ signal: context.signal }); + const base = approvedBase(resolved); + const response = await raceWithSignal( + safeFetch( + `${base}/models`, + { + headers: { + accept: "application/json", + authorization: bearer(resolved, input.id), + ...input.headers?.(resolved), + ...(context.previous?.etag ? { "if-none-match": context.previous.etag } : {}), + }, + signal: context.signal, + }, + { + label: `${input.displayName} catalog endpoint`, + expectedOrigin: new URL(base).origin, + ...(fetchImpl === undefined ? {} : { fetch: fetchImpl }), + }, + ), + context.signal, + ); + if (response.status === 304) + return { + status: "not_modified" as const, + providerId: input.id, + generation: context.generation, + source: { id: `${input.id}-models`, kind: input.sourceKind }, + }; + if (!response.ok) throw new Error(`${input.displayName} catalog returned ${response.status}`); + const body = (await readBoundedJson(response, undefined, context.signal)) as { + data?: unknown[]; + models?: unknown[]; + baseUrl?: unknown; + }; + const rows = body.data ?? body.models; + if (!Array.isArray(rows) || rows.length > MAX_DYNAMIC_MODELS) + throw new TypeError(`${input.displayName} catalog has no bounded model array`); + const endpoint = safeEndpoint(typeof body.baseUrl === "string" ? body.baseUrl : base, { + label: `${input.displayName} model endpoint`, + expectedOrigin: base, + }); + const normalizedRows = rows.map((row) => ({ + raw: row, + model: dynamicModel( + input.id, + endpoint, + input.defaultDialect === undefined || typeof row !== "object" || row === null + ? row + : { + ...(row as Record), + apiDialect: (row as Record).apiDialect ?? input.defaultDialect, + }, + input.modelHeaders, + ), + })); + const imageModels = input.onRows?.(rows, endpoint); + const models = normalizedRows + .filter(({ raw }) => input.rowFilter?.(raw) ?? true) + .map(({ model }) => model); + return { + status: "updated" as const, + providerId: input.id, + generation: context.generation, + source: { id: `${input.id}-models`, kind: input.sourceKind }, + models, + ...(imageModels === undefined ? {} : { imageModels }), + ...(response.headers.get("etag") ? { etag: response.headers.get("etag") as string } : {}), + }; + }, + }); +} + +export function createOpenRouterProvider(options: ProviderFactoryOptions): ModelProvider { + let imageModels: readonly ImageModelInfo[] = []; + const hasOutput = (row: unknown, output: string): boolean => { + if (typeof row !== "object" || row === null || Array.isArray(row)) return false; + const architecture = (row as Record).architecture; + if (typeof architecture !== "object" || architecture === null || Array.isArray(architecture)) + return false; + return ( + Array.isArray((architecture as Record).output_modalities) && + ((architecture as Record).output_modalities as unknown[]).includes(output) + ); + }; + const provider = dynamicProvider({ + id: "openrouter", + displayName: "OpenRouter", + environmentVariables: ["OPENROUTER_API_KEY"], + baseUrl: "https://openrouter.ai/api/v1", + sourceKind: "provider_api", + options, + oauth: createOpenRouterOAuth(options), + defaultDialect: "openai-chat", + rowFilter: (row) => hasOutput(row, "text"), + onRows: (rows, endpoint) => { + imageModels = rows + .filter((row) => hasOutput(row, "image")) + .map((row) => { + const value = row as Record; + const architecture = value.architecture as Record; + if ( + typeof value.id !== "string" || + value.id.length === 0 || + value.id.length > 256 || + typeof value.name !== "string" || + value.name.length === 0 || + value.name.length > MAX_DYNAMIC_NAME_LENGTH + ) { + throw new TypeError("OpenRouter returned invalid image model identity"); + } + return { + providerId: "openrouter", + modelId: value.id, + displayName: value.name, + apiDialect: "openrouter-images", + input: + Array.isArray(architecture.input_modalities) && + architecture.input_modalities.includes("image") + ? ["text", "image"] + : ["text"], + output: ["image"], + endpoint: { type: "fixed", baseUrl: endpoint }, + availability: { status: "available" }, + } satisfies ImageModelInfo; + }); + return imageModels; + }, + }); + const fetchImpl = options.fetch; + return Object.assign(provider, { + listImageModels: () => Promise.resolve(imageModels), + generateImages: async (request: ImageGenerationRequest) => { + const timeout = AbortSignal.timeout(request.timeoutMs ?? 120_000); + const signal = + request.signal === undefined ? timeout : AbortSignal.any([request.signal, timeout]); + const controlledRequest = { ...request, signal }; + const model = imageModels.find((candidate) => candidate.modelId === request.modelId); + if (model === undefined) + throw new TypeError(`OpenRouter has no image model ${request.modelId}`); + const authentication = provider.authentication; + if (authentication === undefined) + throw new AuthError( + "not_configured", + "openrouter", + "OpenRouter authentication is unavailable", + ); + const resolved = await raceWithSignal(authentication.resolve({ signal }), signal); + const encoded = await raceWithSignal( + encodeOpenRouterImageRequest(model, controlledRequest), + signal, + ); + const maximumRetries = Math.min(request.maxRetries ?? 2, 10); + let response: Response | undefined; + for (let attempt = 0; attempt <= maximumRetries; attempt += 1) { + response = await raceWithSignal( + safeFetch( + "https://openrouter.ai/api/v1/images", + { + method: "POST", + headers: { + authorization: bearer(resolved, "openrouter"), + "content-type": "application/json", + }, + body: JSON.stringify(encoded.body), + signal, + }, + { + label: "OpenRouter image endpoint", + expectedOrigin: "https://openrouter.ai", + ...(fetchImpl === undefined ? {} : { fetch: fetchImpl }), + }, + ), + signal, + ); + if ( + response.ok || + attempt === maximumRetries || + ![429, 500, 502, 503, 504].includes(response.status) + ) + break; + await response.body?.cancel(); + const delay = Math.min(250 * 2 ** attempt, request.maxRetryDelayMs ?? 30_000); + await delayWithSignal(delay, signal); + } + if (response === undefined || !response.ok) { + await response?.body?.cancel(); + throw new Error( + `OpenRouter image generation returned ${response?.status ?? "no response"}`, + ); + } + const body = await readBoundedJson(response, 64 * 1024 * 1024, signal); + return decodeOpenRouterImageResponse(body, { + model, + request: controlledRequest, + secretValues: resolved.secretValues, + }); + }, + }); +} + +export const createGitHubCopilotProvider = (options: ProviderFactoryOptions): ModelProvider => { + const requiredHeaders = { + "copilot-integration-id": "vscode-chat", + "editor-version": "vscode/1.107.0", + "editor-plugin-version": "copilot-chat/0.35.0", + } as const; + return dynamicProvider({ + id: "github-copilot", + displayName: "GitHub Copilot", + environmentVariables: ["COPILOT_GITHUB_TOKEN"], + baseUrl: "https://api.individual.githubcopilot.com", + sourceKind: "entitlement", + options, + oauth: createGitHubCopilotOAuth(options), + apiKey: createGitHubCopilotTokenAuth(options), + endpoint: (resolved) => resolved.auth.baseUrl ?? "https://api.individual.githubcopilot.com", + allowEndpoint: (url) => + url.protocol === "https:" && + (url.hostname === "api.individual.githubcopilot.com" || + url.hostname.endsWith(".githubcopilot.com")), + headers: () => requiredHeaders, + modelHeaders: requiredHeaders, + }); +}; + +export function createCloudflareAiGatewayProvider(options: ProviderFactoryOptions): ModelProvider { + const id = "cloudflare-ai-gateway"; + const method = configuredApiKey({ + providerId: id, + displayName: "Cloudflare AI Gateway token", + keyEnv: "CLOUDFLARE_API_KEY", + settings: [ + { env: "CLOUDFLARE_ACCOUNT_ID", key: "Cloudflare account ID" }, + { env: "CLOUDFLARE_GATEWAY_ID", key: "Cloudflare gateway ID" }, + ], + }); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file"], + methods: { apiKey: method }, + store: options.store, + context: options.context, + }); + const fetchImpl = options.fetch; + const base = (resolved: ResolvedAuth) => + `https://gateway.ai.cloudflare.com/v1/${encodeURIComponent(resolved.env?.CLOUDFLARE_ACCOUNT_ID ?? "")}/${encodeURIComponent(resolved.env?.CLOUDFLARE_GATEWAY_ID ?? "")}/compat`; + const provider = new HttpSseProvider({ + id, + displayName: "Cloudflare AI Gateway", + authMethods: authentication.methods, + authentication, + models: [], + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: codecs(id, { anthropicBearer: true }), + validateEndpoint: (url, _model, resolved) => { + if (url.origin !== new URL(base(resolved)).origin) + throw new TypeError("Cloudflare AI Gateway request endpoint has an unapproved origin"); + }, + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + }); + return Object.assign(provider, { + refreshModelCatalog: async (context: ModelCatalogRefreshContext) => { + const resolved = await authentication.resolve({ signal: context.signal }); + const endpoint = base(resolved); + const response = await raceWithSignal( + safeFetch( + `${endpoint}/models`, + { + headers: { authorization: bearer(resolved, id), accept: "application/json" }, + signal: context.signal, + }, + { + label: "Cloudflare AI Gateway catalog endpoint", + expectedOrigin: new URL(endpoint).origin, + ...(fetchImpl === undefined ? {} : { fetch: fetchImpl }), + }, + ), + context.signal, + ); + if (!response.ok) + throw new Error(`Cloudflare AI Gateway catalog returned ${response.status}`); + const body = (await readBoundedJson(response, undefined, context.signal)) as { + data?: unknown[]; + }; + if (!Array.isArray(body.data) || body.data.length > MAX_DYNAMIC_MODELS) + throw new TypeError("Cloudflare AI Gateway catalog has no bounded model array"); + const models = body.data.map((row) => dynamicModel(id, endpoint, row)); + return { + status: "updated" as const, + providerId: id, + generation: context.generation, + source: { id: "cloudflare-ai-gateway-models", kind: "gateway" as const }, + models, + }; + }, + }); +} + +export function createRadiusProvider( + options: ProviderFactoryOptions & { baseUrl?: string }, +): ModelProvider { + const id = "radius"; + const gateway = safeEndpoint(options.baseUrl ?? "https://radius.pi.dev", { + label: "Radius gateway endpoint", + allowLoopbackHttp: options.baseUrl !== undefined, + }); + const method = createEnvironmentApiKeyAuth({ + providerId: id, + displayName: "Radius API key", + environmentVariables: ["RADIUS_API_KEY"], + }); + const authentication = createProviderAuthentication({ + providerId: id, + declaredMethods: ["environment", "file", "oauth"], + methods: { apiKey: method, oauth: createRadiusOAuth(gateway, options) }, + store: options.store, + context: options.context, + }); + const fetchImpl = options.fetch; + const provider = new HttpSseProvider({ + id, + displayName: "Radius", + allowLoopbackHttp: options.baseUrl !== undefined, + authMethods: authentication.methods, + authentication, + models: [], + resolveAuth: (signal) => authentication.resolve({ signal }), + codecFor: codecs(id), + validateEndpoint: (url) => { + if (url.origin !== new URL(gateway).origin) + throw new TypeError("Radius request endpoint has an unapproved origin"); + }, + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + }); + return Object.assign(provider, { + refreshModelCatalog: async (context: ModelCatalogRefreshContext) => { + const resolved = await authentication.resolve({ signal: context.signal }); + const response = await raceWithSignal( + safeFetch( + `${gateway}/v1/config`, + { + headers: { accept: "application/json", authorization: bearer(resolved, id) }, + signal: context.signal, + }, + { + label: "Radius catalog endpoint", + allowLoopbackHttp: options.baseUrl !== undefined, + expectedOrigin: new URL(gateway).origin, + ...(fetchImpl === undefined ? {} : { fetch: fetchImpl }), + }, + ), + context.signal, + ); + if (!response.ok) throw new Error(`Radius catalog returned ${response.status}`); + const body = (await readBoundedJson(response, undefined, context.signal)) as { + baseUrl?: unknown; + models?: unknown[]; + }; + if ( + typeof body.baseUrl !== "string" || + !Array.isArray(body.models) || + body.models.length > MAX_DYNAMIC_MODELS + ) { + throw new TypeError("Radius config is malformed or exceeds model limits"); + } + const endpoint = safeEndpoint(body.baseUrl, { + label: "Radius model endpoint", + allowLoopbackHttp: options.baseUrl !== undefined, + expectedOrigin: gateway, + }); + const models = body.models.map((row) => { + if (typeof row !== "object" || row === null || Array.isArray(row)) + throw new TypeError("Radius returned a malformed model"); + const value = row as Record; + if ( + typeof value.reasoning !== "boolean" || + typeof value.toolUse !== "boolean" || + !Array.isArray(value.input) || + !value.input.every((item) => typeof item === "string") + ) { + throw new TypeError("Radius returned incomplete capability metadata"); + } + return dynamicModel(id, endpoint, { + ...value, + apiDialect: "gateway-messages", + context_length: value.contextWindow, + maxOutputTokens: value.maxTokens, + architecture: { input_modalities: value.input }, + supported_parameters: [ + ...(value.toolUse ? ["tools"] : []), + ...(value.reasoning ? ["reasoning"] : []), + ], + }); + }); + return { + status: "updated" as const, + providerId: id, + generation: context.generation, + source: { id: "radius-config", kind: "gateway" as const }, + models, + }; + }, + }); +} + +export interface CustomProviderConfiguration { + readonly baseUrl: string; + readonly models: readonly ModelInfo[]; + readonly headers?: Readonly>; + readonly apiKeyEnvironmentVariables?: readonly string[]; +} + +export interface CustomProviderOptions + extends ProviderFactoryOptions, + Partial { + readonly id?: string; + readonly displayName?: string; +} + +export function parseCustomProviderConfiguration( + value: unknown, + providerId = "custom", +): CustomProviderConfiguration { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new TypeError("Custom provider configuration must be an object"); + } + const input = value as Record; + const allowed = new Set(["baseUrl", "models", "headers", "apiKeyEnvironmentVariables"]); + if (Object.keys(input).some((key) => !allowed.has(key))) { + throw new TypeError("Custom provider configuration contains an unknown field"); + } + if ( + typeof input.baseUrl !== "string" || + !Array.isArray(input.models) || + input.models.length === 0 + ) { + throw new TypeError("Custom provider configuration requires baseUrl and at least one model"); + } + if ( + input.headers !== undefined && + (typeof input.headers !== "object" || + input.headers === null || + Array.isArray(input.headers) || + Object.values(input.headers).some((header) => typeof header !== "string")) + ) { + throw new TypeError("Custom provider headers must contain only string values"); + } + if ( + input.apiKeyEnvironmentVariables !== undefined && + (!Array.isArray(input.apiKeyEnvironmentVariables) || + input.apiKeyEnvironmentVariables.some( + (name) => typeof name !== "string" || !/^[A-Z_][A-Z0-9_]*$/.test(name), + ) || + new Set(input.apiKeyEnvironmentVariables).size !== input.apiKeyEnvironmentVariables.length) + ) { + throw new TypeError("Custom provider credential environment variables are invalid"); + } + const models = input.models.map((model) => ({ + ...(model as ModelInfo), + providerId, + })); + return { + baseUrl: safeEndpoint(input.baseUrl, { + label: "User configured endpoint", + allowLoopbackHttp: true, + }), + models, + ...(input.headers === undefined + ? {} + : { headers: { ...(input.headers as Readonly>) } }), + ...(input.apiKeyEnvironmentVariables === undefined + ? {} + : { apiKeyEnvironmentVariables: [...(input.apiKeyEnvironmentVariables as string[])] }), + }; +} + +export function createCustomProvider(options: CustomProviderOptions): ModelProvider { + const id = options.id ?? "custom"; + const displayName = options.displayName ?? id; + const models = options.models ?? []; + if (models.length === 0) { + return { + id, + displayName, + authMethods: ["keyless"], + listModels: () => Promise.resolve([]), + stream: async function* () { + yield { + type: "error", + code: "custom_not_configured", + message: "User configured endpoint has no models", + retryable: false, + category: "invalid_request", + requestPhase: "before_dispatch", + }; + }, + }; + } + const baseUrl = options.baseUrl; + if (!baseUrl) throw new TypeError("User configured endpoint requires baseUrl"); + const endpoint = safeEndpoint(baseUrl, { + label: "User configured endpoint", + allowLoopbackHttp: true, + }); + const supportedDialects = new Set([ + "openai-chat", + "openai-responses", + "anthropic-messages", + "google-generative-ai", + "mistral-conversations", + "gateway-messages", + ]); + if (models.some((model) => !supportedDialects.has(model.apiDialect))) { + throw new TypeError("User configured endpoint contains an unsupported API dialect"); + } + const normalized = models.map((model) => ({ + ...model, + providerId: id, + endpoint: { type: "fixed", baseUrl: endpoint } as const, + headers: { ...model.headers, ...options.headers }, + })); + validateModelCatalog(normalized); + if (normalized.some((model) => model.compatibility?.dialect !== model.apiDialect)) { + throw new TypeError("User configured models require an explicit matching compatibility record"); + } + if ((options.apiKeyEnvironmentVariables?.length ?? 0) === 0) { + return new HttpSseProvider({ + id, + displayName, + allowLoopbackHttp: true, + authMethods: ["keyless"], + models: normalized, + resolveAuth: () => Promise.resolve({ auth: {}, source: "keyless", secretValues: [] }), + codecFor: codecs(id, { keyless: true }), + validateEndpoint: (url) => { + if (url.origin !== new URL(endpoint).origin) + throw new TypeError("User configured request endpoint changed origin"); + }, + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + }); + } + return apiKeyProvider({ + allowLoopbackHttp: true, + id, + displayName, + environmentVariables: options.apiKeyEnvironmentVariables ?? [], + options, + models: normalized, + validateEndpoint: (url) => { + if (url.origin !== new URL(endpoint).origin) + throw new TypeError("User configured request endpoint changed origin"); + }, + }); +} diff --git a/packages/ai/src/request-preparation.ts b/packages/ai/src/request-preparation.ts new file mode 100644 index 0000000..12f5f91 --- /dev/null +++ b/packages/ai/src/request-preparation.ts @@ -0,0 +1,1274 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createHash } from "node:crypto"; + +import type { BlobReference, JsonObject, ThinkingLevel } from "@axl/protocol"; + +import { assertModelSupports } from "./capabilities.ts"; +import { + FrozenToolRoster, + GENERIC_TOOL_DIALECT, + OPENAI_CHAT_TOOL_DIALECT, + renderToolName, + type ToolDialectData, +} from "./dialect.ts"; +import type { + ApiDialect, + CacheRetention, + ModelInfo, + ModelRequest, + ProviderContinuationMetadata, + ProviderModelIdentity, + ProviderSignature, + RequestAssistantContent, + RequestModelMessage, + RequestToolCall, + RequestToolDeclaration, + SamplingOptionName, + SamplingOptions, +} from "./model.ts"; +import { clampThinkingLevel, fitThinkingBudget } from "./thinking.ts"; + +const FORBIDDEN_METADATA_KEYS = [ + "authorization", + "cookie", + "credential", + "password", + "secret", + "token", + "apikey", +] as const; +const PORTABLE_TOOL_CALL_ID = /^[A-Za-z0-9_-]{1,64}$/; +const MISTRAL_TOOL_CALL_ID = /^[A-Za-z0-9]{9}$/; + +export class RequestPreparationError extends Error { + readonly path: string; + + constructor(path: string, message: string) { + super(`${path} ${message}`); + this.name = "RequestPreparationError"; + this.path = path; + } +} + +export interface PreparedBlob { + readonly reference: BlobReference; + readonly bytes: Uint8Array; +} + +export type PreparedToolConstraint = + | { readonly type: "json-schema"; readonly strict: boolean } + | { + readonly type: "grammar"; + readonly format: "lark" | "regex"; + readonly definition: string; + readonly inputProperty: string; + }; + +export interface PreparedToolDeclaration extends RequestToolDeclaration { + readonly canonicalName: string; + readonly preparedConstraint?: PreparedToolConstraint; +} + +export interface PreparedToolCall extends RequestToolCall { + readonly canonicalCallId: string; + readonly canonicalName: string; +} + +export type PreparedRequestMessage = + | Extract + | (Omit, "toolCalls"> & { + readonly toolCalls?: readonly PreparedToolCall[]; + }) + | (Extract & { + readonly canonicalCallId: string; + readonly canonicalName: string; + }); + +export interface CachePlacement { + readonly target: "system" | "tool" | "message-content"; + readonly toolIndex?: number; + readonly messageIndex?: number; + readonly contentIndex?: number; +} + +export interface PreparedReasoning { + readonly requested: ThinkingLevel; + readonly effective: ThinkingLevel; + readonly clamped: boolean; + readonly providerValue?: string; + readonly tokenBudget?: number; + readonly maxTokens?: number; +} + +export interface RequestSanitization { + readonly path: string; + readonly reason: "foreign-provider-signature" | "foreign-provider-continuation"; +} + +export interface RequestPreparation { + readonly blobs: ReadonlyMap; + readonly tools: readonly PreparedToolDeclaration[]; + readonly reasoning?: PreparedReasoning; + readonly cache: { + readonly retention: CacheRetention; + readonly sessionId?: string; + readonly placements: readonly CachePlacement[]; + }; + readonly sanitizations: readonly RequestSanitization[]; +} + +export interface PreparedModelRequest extends Omit { + readonly messages: readonly PreparedRequestMessage[]; + readonly tools?: readonly PreparedToolDeclaration[]; + readonly preparation: RequestPreparation; +} + +function fail(path: string, message: string): never { + throw new RequestPreparationError(path, message); +} + +function exactKeys(value: object, allowed: readonly string[], path: string): void { + const accepted = new Set(allowed); + for (const key of Object.keys(value)) { + if (!accepted.has(key)) fail(`${path}.${key}`, "is not supported"); + } +} + +function nonEmpty(value: unknown, path: string): asserts value is string { + if (typeof value !== "string" || value.trim().length === 0) + fail(path, "must be a non-empty string"); +} + +function finite(value: number | undefined, path: string, minimum: number, maximum: number): void { + if (value !== undefined && (!Number.isFinite(value) || value < minimum || value > maximum)) { + fail(path, `must be between ${minimum} and ${maximum}`); + } +} + +function forbiddenMetadataKey(key: string): boolean { + const normalized = key.replace(/[^a-z0-9]/gi, "").toLowerCase(); + return FORBIDDEN_METADATA_KEYS.some( + (forbidden) => normalized === forbidden || normalized.endsWith(forbidden), + ); +} + +function validateJson( + value: unknown, + path: string, + ancestors = new Set(), + rejectAuthorizationData = false, +): void { + if ( + value === null || + typeof value === "string" || + typeof value === "boolean" || + (typeof value === "number" && Number.isFinite(value)) + ) { + return; + } + if (typeof value !== "object") fail(path, "must be JSON-compatible"); + if (ancestors.has(value)) fail(path, "must not contain cycles"); + const next = new Set(ancestors); + next.add(value); + if (Array.isArray(value)) { + value.forEach((item, index) => { + validateJson(item, `${path}[${index}]`, next, rejectAuthorizationData); + }); + return; + } + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) fail(path, "must be a plain object"); + for (const [key, item] of Object.entries(value)) { + if (rejectAuthorizationData && forbiddenMetadataKey(key)) { + fail(`${path}.${key}`, "must not contain authorization data"); + } + validateJson(item, `${path}.${key}`, next, rejectAuthorizationData); + } +} + +function validateMetadata(metadata: ModelRequest["metadata"]): void { + if (metadata === undefined) return; + if (typeof metadata !== "object" || metadata === null || Array.isArray(metadata)) { + fail("request.metadata", "must be an object"); + } + validateJson(metadata, "request.metadata", new Set(), true); +} + +const GOOGLE_SAFETY_CATEGORIES = new Set([ + "HARM_CATEGORY_HARASSMENT", + "HARM_CATEGORY_HATE_SPEECH", + "HARM_CATEGORY_SEXUALLY_EXPLICIT", + "HARM_CATEGORY_DANGEROUS_CONTENT", + "HARM_CATEGORY_CIVIC_INTEGRITY", +]); +const GOOGLE_SAFETY_THRESHOLDS = new Set([ + "BLOCK_NONE", + "BLOCK_LOW_AND_ABOVE", + "BLOCK_MEDIUM_AND_ABOVE", + "BLOCK_ONLY_HIGH", + "OFF", +]); + +function prepareSafetySettings( + model: ModelInfo, + settings: ModelRequest["safetySettings"], +): ModelRequest["safetySettings"] { + if (settings === undefined) return undefined; + if (model.apiDialect !== "google-generative-ai") { + fail("request.safetySettings", "is unsupported by this model dialect"); + } + if (!Array.isArray(settings)) fail("request.safetySettings", "must be an array"); + const categories = new Set(); + return Object.freeze( + settings.map((setting, index) => { + const path = `request.safetySettings[${index}]`; + if (typeof setting !== "object" || setting === null || Array.isArray(setting)) { + fail(path, "must be an object"); + } + exactKeys(setting, ["category", "threshold"], path); + if (!GOOGLE_SAFETY_CATEGORIES.has(setting.category)) { + fail(`${path}.category`, "is not recognized"); + } + if (!GOOGLE_SAFETY_THRESHOLDS.has(setting.threshold)) { + fail(`${path}.threshold`, "is not recognized"); + } + if (categories.has(setting.category)) fail(`${path}.category`, "is duplicated"); + categories.add(setting.category); + return Object.freeze({ category: setting.category, threshold: setting.threshold }); + }), + ); +} + +const DEFAULT_SAMPLING_SUPPORT: Readonly> = { + "openai-chat": ["temperature", "topP", "frequencyPenalty", "presencePenalty", "seed"], + "openai-responses": ["temperature", "topP"], + "azure-openai-responses": ["temperature", "topP"], + "openai-codex-responses": ["temperature", "topP"], + "anthropic-messages": ["temperature", "topP", "topK"], + "google-generative-ai": ["temperature", "topP", "topK", "seed"], + "google-vertex": ["temperature", "topP", "topK", "seed"], + "bedrock-converse-stream": ["temperature", "topP"], + "mistral-conversations": ["temperature", "topP", "frequencyPenalty", "presencePenalty", "seed"], + "gateway-messages": [], + fake: [ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", + ], +}; + +function prepareSampling( + model: ModelInfo, + sampling: SamplingOptions | undefined, +): SamplingOptions | undefined { + if (sampling === undefined) return undefined; + if (typeof sampling !== "object" || sampling === null || Array.isArray(sampling)) { + fail("request.sampling", "must be an object"); + } + exactKeys( + sampling, + [ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", + "custom", + ], + "request.sampling", + ); + finite(sampling.temperature, "request.sampling.temperature", 0, 2); + finite(sampling.topP, "request.sampling.topP", 0, 1); + finite(sampling.minP, "request.sampling.minP", 0, 1); + finite(sampling.frequencyPenalty, "request.sampling.frequencyPenalty", -2, 2); + finite(sampling.presencePenalty, "request.sampling.presencePenalty", -2, 2); + finite(sampling.repetitionPenalty, "request.sampling.repetitionPenalty", 0, Number.MAX_VALUE); + if (sampling.topK !== undefined && (!Number.isSafeInteger(sampling.topK) || sampling.topK < 1)) { + fail("request.sampling.topK", "must be a positive safe integer"); + } + if (sampling.seed !== undefined && !Number.isSafeInteger(sampling.seed)) { + fail("request.sampling.seed", "must be a safe integer"); + } + if (sampling.custom !== undefined) { + validateJson(sampling.custom, "request.sampling.custom", new Set(), true); + } + const supported = new Set( + model.sampling?.supported ?? DEFAULT_SAMPLING_SUPPORT[model.apiDialect] ?? [], + ); + for (const option of [ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", + ] as const) { + if (sampling[option] !== undefined && !supported.has(option)) { + fail(`request.sampling.${option}`, "is unsupported by this model"); + } + } + if (sampling.custom !== undefined) { + const customFields = new Set(model.sampling?.customFields ?? []); + for (const field of Object.keys(sampling.custom)) { + if (!customFields.has(field)) + fail(`request.sampling.custom.${field}`, "is unsupported by this model"); + } + } + return Object.freeze({ + ...sampling, + ...(sampling.custom === undefined ? {} : { custom: structuredClone(sampling.custom) }), + }); +} + +function targetIdentity(model: ModelInfo): ProviderModelIdentity { + return { providerId: model.providerId, apiDialect: model.apiDialect, modelId: model.modelId }; +} + +function matchesTarget(metadata: ProviderModelIdentity, target: ProviderModelIdentity): boolean { + return ( + metadata.providerId === target.providerId && + metadata.apiDialect === target.apiDialect && + metadata.modelId === target.modelId + ); +} + +function validateIdentity(value: ProviderModelIdentity, path: string): void { + exactKeys(value, ["providerId", "apiDialect", "modelId"], path); + nonEmpty(value.providerId, `${path}.providerId`); + nonEmpty(value.apiDialect, `${path}.apiDialect`); + nonEmpty(value.modelId, `${path}.modelId`); +} + +function keepSignature( + signature: ProviderSignature | undefined, + target: ProviderModelIdentity, + path: string, + sanitizations: RequestSanitization[], +): ProviderSignature | undefined { + if (signature === undefined) return undefined; + exactKeys(signature, ["providerId", "apiDialect", "modelId", "value"], path); + nonEmpty(signature.providerId, `${path}.providerId`); + nonEmpty(signature.apiDialect, `${path}.apiDialect`); + nonEmpty(signature.modelId, `${path}.modelId`); + nonEmpty(signature.value, `${path}.value`); + if (matchesTarget(signature, target)) return Object.freeze({ ...signature }); + sanitizations.push({ path, reason: "foreign-provider-signature" }); + return undefined; +} + +function keepContinuation( + continuation: ProviderContinuationMetadata | undefined, + target: ProviderModelIdentity, + path: string, + sanitizations: RequestSanitization[], +): ProviderContinuationMetadata | undefined { + if (continuation === undefined) return undefined; + exactKeys( + continuation, + ["providerId", "apiDialect", "modelId", "responseId", "itemId", "namespace"], + path, + ); + nonEmpty(continuation.providerId, `${path}.providerId`); + nonEmpty(continuation.apiDialect, `${path}.apiDialect`); + nonEmpty(continuation.modelId, `${path}.modelId`); + for (const [key, value] of Object.entries(continuation)) { + if (key === "providerId" || key === "apiDialect" || key === "modelId") continue; + if (value !== undefined) nonEmpty(value, `${path}.${key}`); + } + if (matchesTarget(continuation, target)) return Object.freeze({ ...continuation }); + sanitizations.push({ path, reason: "foreign-provider-continuation" }); + return undefined; +} + +function toolCallIdRule(apiDialect: ApiDialect): RegExp { + return apiDialect === "mistral-conversations" ? MISTRAL_TOOL_CALL_ID : PORTABLE_TOOL_CALL_ID; +} + +function normalizedToolCallId(id: string, apiDialect: ApiDialect, occupied: Set): string { + const rule = toolCallIdRule(apiDialect); + if (rule.test(id) && !occupied.has(id)) return id; + const digest = createHash("sha256").update(id).digest("hex"); + const prefix = apiDialect === "mistral-conversations" ? "" : "call_"; + const length = apiDialect === "mistral-conversations" ? 9 : 64; + for (let counter = 0; counter < 1000; counter += 1) { + const candidateDigest = + counter === 0 ? digest : createHash("sha256").update(`${id}:${counter}`).digest("hex"); + const candidate = `${prefix}${candidateDigest}`.slice(0, length); + if (!occupied.has(candidate)) return candidate; + } + return fail("request.messages", "contains too many colliding tool call identifiers"); +} + +function toolDialectFor(apiDialect: ApiDialect): ToolDialectData { + if ( + apiDialect === "openai-chat" || + apiDialect === "openai-responses" || + apiDialect === "azure-openai-responses" || + apiDialect === "openai-codex-responses" || + apiDialect === "anthropic-messages" || + apiDialect === "bedrock-converse-stream" + ) { + return { ...OPENAI_CHAT_TOOL_DIALECT, id: apiDialect }; + } + return { ...GENERIC_TOOL_DIALECT, id: apiDialect }; +} + +interface JsonSchemaNode { + [key: string]: unknown; + type?: unknown; + properties?: Record; + required?: unknown; +} + +const UNSUPPORTED_STRICT_KEYS = new Set([ + "$ref", + "$defs", + "definitions", + "allOf", + "oneOf", + "patternProperties", + "dependentSchemas", + "dependencies", + "unevaluatedProperties", + "propertyNames", + "contains", + "prefixItems", + "not", + "if", + "then", + "else", +]); + +function schemaObject(value: unknown): value is JsonSchemaNode { + return typeof value === "object" && value !== null && !Array.isArray(value); +} + +function structuredSchema(schema: unknown): boolean { + if (!schemaObject(schema)) return false; + const types = + typeof schema.type === "string" ? [schema.type] : Array.isArray(schema.type) ? schema.type : []; + return ( + types.includes("object") || + types.includes("array") || + schema.properties !== undefined || + schema.items !== undefined + ); +} + +function schemaAllowsNull(schema: unknown): boolean { + if (!schemaObject(schema)) return false; + if (schema.type === "null" || (Array.isArray(schema.type) && schema.type.includes("null"))) + return true; + if (schema.const === null || (Array.isArray(schema.enum) && schema.enum.includes(null))) + return true; + return Array.isArray(schema.anyOf) && schema.anyOf.some(schemaAllowsNull); +} + +function makeStrictNode(schema: unknown, path: string): void { + if (!schemaObject(schema)) fail(path, "uses an unsupported boolean schema"); + for (const key of Object.keys(schema)) { + if (UNSUPPORTED_STRICT_KEYS.has(key)) + fail(`${path}.${key}`, "is unsupported in strict schemas"); + } + if (schema.anyOf !== undefined) { + if (!Array.isArray(schema.anyOf) || schema.anyOf.length === 0) + fail(`${path}.anyOf`, "must not be empty"); + schema.anyOf.forEach((variant, index) => { + if (structuredSchema(variant)) { + fail(`${path}.anyOf[${index}]`, "uses an unsupported structured union"); + } + makeStrictNode(variant, `${path}.anyOf[${index}]`); + }); + } + if (schema.items !== undefined) { + if (Array.isArray(schema.items)) + fail(`${path}.items`, "tuple schemas are unsupported in strict mode"); + makeStrictNode(schema.items, `${path}.items`); + } + if (schema.type !== "object") { + if (schema.properties !== undefined) fail(`${path}.properties`, "requires type object"); + return; + } + if (schema.additionalProperties !== undefined && schema.additionalProperties !== false) { + fail(`${path}.additionalProperties`, "must be false in strict mode"); + } + if (schema.properties !== undefined && !schemaObject(schema.properties)) { + fail(`${path}.properties`, "must be a schema map"); + } + if ( + schema.required !== undefined && + (!Array.isArray(schema.required) || schema.required.some((key) => typeof key !== "string")) + ) { + fail(`${path}.required`, "must be a string array"); + } + const properties = schema.properties ?? {}; + const names = Object.keys(properties); + const requiredValues = Array.isArray(schema.required) ? schema.required : []; + const required = new Set(requiredValues); + if (required.size !== requiredValues.length) + fail(`${path}.required`, "must not contain duplicates"); + for (const name of required) { + if (!names.includes(name)) + fail(`${path}.required`, `contains unknown property ${String(name)}`); + } + for (const [name, property] of Object.entries(properties)) { + makeStrictNode(property, `${path}.properties.${name}`); + if (!required.has(name) && !schemaAllowsNull(property)) { + properties[name] = { anyOf: [property, { type: "null" }] }; + } + } + schema.required = names; + schema.additionalProperties = false; +} + +function strictSchema(schema: JsonObject, path: string): JsonObject { + validateJson(schema, path); + const clone = structuredClone(schema) as JsonObject; + makeStrictNode(clone, path); + if (clone.type !== "object") fail(path, "must have object at its root for strict mode"); + return clone; +} + +function strictToolSupport(model: ModelInfo): boolean { + const compatibility = model.compatibility; + if (compatibility === undefined) return false; + if ( + compatibility.dialect === "openai-chat" || + compatibility.dialect === "openai-responses" || + compatibility.dialect === "azure-openai-responses" || + compatibility.dialect === "openai-codex-responses" || + compatibility.dialect === "anthropic-messages" || + compatibility.dialect === "google-generative-ai" || + compatibility.dialect === "google-vertex" || + compatibility.dialect === "bedrock-converse-stream" || + compatibility.dialect === "mistral-conversations" || + compatibility.dialect === "gateway-messages" + ) { + return compatibility.supportsStrictTools === true; + } + return false; +} + +function grammarToolSupport(model: ModelInfo): boolean { + const compatibility = model.compatibility; + return ( + compatibility !== undefined && + (compatibility.dialect === "openai-chat" || + compatibility.dialect === "openai-responses" || + compatibility.dialect === "azure-openai-responses" || + compatibility.dialect === "openai-codex-responses") && + compatibility.supportsGrammarTools === true + ); +} + +function grammarInputProperty(schema: JsonObject, path: string): string { + if (schema.type !== "object") fail(path, "requires an object schema"); + if (!Array.isArray(schema.required) || schema.required.length !== 1) { + fail(path, "requires exactly one required string property"); + } + const property = schema.required[0]; + if (typeof property !== "string") fail(`${path}.required[0]`, "must be a string"); + const properties = schema.properties; + if (!schemaObject(properties) || !schemaObject(properties[property])) { + fail(path, `requires a schema for property ${property}`); + } + if (properties[property].type !== "string") fail(path, `requires ${property} to be a string`); + return property; +} + +function prepareTools( + model: ModelInfo, + requestTools: readonly RequestToolDeclaration[] | undefined, + dialect?: ToolDialectData, +): readonly PreparedToolDeclaration[] { + if (requestTools === undefined || requestTools.length === 0) return Object.freeze([]); + const roster = new FrozenToolRoster(dialect ?? toolDialectFor(model.apiDialect), requestTools); + return Object.freeze( + roster.tools.map((visible, index) => { + const source = requestTools[index]; + if (source === undefined) return fail(`request.tools[${index}]`, "is missing"); + const path = `request.tools[${index}]`; + exactKeys(source, ["name", "description", "inputSchema", "constraint"], path); + nonEmpty(source.name, `${path}.name`); + nonEmpty(source.description, `${path}.description`); + if (source.constraint !== undefined) { + if (source.constraint.type !== "grammar" && source.constraint.type !== "json-schema") { + fail(`${path}.constraint.type`, "is not recognized"); + } + exactKeys( + source.constraint, + source.constraint.type === "grammar" ? ["type", "variants"] : ["type", "strict"], + `${path}.constraint`, + ); + if (source.constraint.type === "grammar") { + if ( + typeof source.constraint.variants !== "object" || + source.constraint.variants === null || + Array.isArray(source.constraint.variants) + ) { + fail(`${path}.constraint.variants`, "must be an object"); + } + exactKeys(source.constraint.variants, ["lark", "regex"], `${path}.constraint.variants`); + for (const [format, definition] of Object.entries(source.constraint.variants)) { + if (typeof definition !== "string") { + fail(`${path}.constraint.variants.${format}`, "must be a string"); + } + } + } else if ( + source.constraint.strict !== "prefer" && + source.constraint.strict !== "require" + ) { + fail(`${path}.constraint.strict`, "must be prefer or require"); + } + } + validateJson(visible.inputSchema, `${path}.inputSchema`); + let inputSchema = structuredClone(visible.inputSchema); + let constraint: PreparedToolConstraint | undefined; + if (source.constraint?.type === "json-schema") { + if (!strictToolSupport(model)) { + if (source.constraint.strict === "require") { + fail( + `request.tools[${index}].constraint`, + "requires strict schemas unsupported by this model", + ); + } + constraint = { type: "json-schema", strict: false }; + } else { + try { + inputSchema = strictSchema(inputSchema, `request.tools[${index}].inputSchema`); + constraint = { type: "json-schema", strict: true }; + } catch (error) { + if (source.constraint.strict === "require") throw error; + constraint = { type: "json-schema", strict: false }; + } + } + } else if (source.constraint?.type === "grammar") { + if (!grammarToolSupport(model)) { + fail( + `request.tools[${index}].constraint`, + "requires grammar tools unsupported by this model", + ); + } + const lark = source.constraint.variants.lark?.trim(); + const regex = source.constraint.variants.regex?.trim(); + if (!lark && !regex) + fail(`request.tools[${index}].constraint`, "has no usable grammar variant"); + constraint = { + type: "grammar", + format: lark ? "lark" : "regex", + definition: lark ?? (regex as string), + inputProperty: grammarInputProperty(inputSchema, `request.tools[${index}].inputSchema`), + }; + } + return Object.freeze({ + name: visible.name, + canonicalName: visible.canonicalName, + description: visible.description, + inputSchema, + ...(source.constraint === undefined + ? {} + : { constraint: structuredClone(source.constraint) }), + ...(constraint === undefined ? {} : { preparedConstraint: constraint }), + }); + }), + ); +} + +function prepareReasoning(model: ModelInfo, request: ModelRequest): PreparedReasoning | undefined { + if (request.thinkingLevel === undefined) return undefined; + const clamp = clampThinkingLevel(model, request.thinkingLevel); + if (clamp.effective === "off") return Object.freeze(clamp); + const providerValue = model.thinkingLevelMap?.[clamp.effective]; + const compatibility = model.compatibility; + const googleBudget = + compatibility?.dialect === "google-generative-ai" && + typeof providerValue === "string" && + /^\d+$/.test(providerValue) + ? Number.parseInt(providerValue, 10) + : undefined; + const usesBudget = + (compatibility?.dialect === "openai-chat" && + compatibility.thinkingTokenBudgetField !== undefined) || + (compatibility?.dialect === "anthropic-messages" && + compatibility.forceAdaptiveThinking !== true) || + (compatibility?.dialect === "bedrock-converse-stream" && + compatibility.supportsThinkingSignatures === true && + compatibility.forceAdaptiveThinking !== true) || + googleBudget !== undefined; + if (!usesBudget) { + return Object.freeze({ + ...clamp, + ...(providerValue === undefined || providerValue === null ? {} : { providerValue }), + }); + } + const budgetLevel = + clamp.effective === "xhigh" || clamp.effective === "max" ? "high" : clamp.effective; + const budgets = + googleBudget === undefined + ? request.thinkingBudgets + : { + ...request.thinkingBudgets, + [budgetLevel]: request.thinkingBudgets?.[budgetLevel] ?? googleBudget, + }; + const fitted = fitThinkingBudget({ + level: clamp.effective, + modelMaxTokens: model.maxOutputTokens, + ...(request.maxOutputTokens === undefined + ? {} + : { requestedMaxTokens: request.maxOutputTokens }), + ...(budgets === undefined ? {} : { budgets }), + }); + const prepared = { + ...clamp, + ...(compatibility?.dialect !== "google-generative-ai" || + providerValue === undefined || + providerValue === null + ? {} + : { providerValue }), + tokenBudget: fitted.thinkingBudget, + }; + Object.defineProperty(prepared, "maxTokens", { + value: fitted.maxTokens, + enumerable: false, + }); + return Object.freeze(prepared); +} + +function resolvedMaxOutputTokens( + model: ModelInfo, + request: ModelRequest, + reasoning: PreparedReasoning | undefined, +): number | undefined { + const requested = request.maxOutputTokens; + if (requested !== undefined && (!Number.isSafeInteger(requested) || requested < 1)) { + fail("request.maxOutputTokens", "must be a positive safe integer"); + } + if (requested !== undefined && requested > model.maxOutputTokens) { + fail("request.maxOutputTokens", `exceeds model limit ${model.maxOutputTokens}`); + } + if (reasoning?.tokenBudget === undefined) return requested; + return reasoning.maxTokens ?? model.maxOutputTokens; +} + +function prepareCache( + model: ModelInfo, + request: ModelRequest, + tools: readonly PreparedToolDeclaration[], + messages: readonly PreparedRequestMessage[], +): RequestPreparation["cache"] { + if (request.cache !== undefined) { + if ( + typeof request.cache !== "object" || + request.cache === null || + Array.isArray(request.cache) + ) { + fail("request.cache", "must be an object"); + } + exactKeys(request.cache, ["retention", "sessionId"], "request.cache"); + if ( + request.cache.retention !== undefined && + request.cache.retention !== "none" && + request.cache.retention !== "short" && + request.cache.retention !== "long" + ) { + fail("request.cache.retention", "is not recognized"); + } + } + const retention = request.cache?.retention ?? model.cache?.defaultRetention ?? "none"; + if (retention !== "none") { + if (model.cache?.supported !== true) + fail("request.cache.retention", "is unsupported by this model"); + if (!model.cache.supportedRetentions.includes(retention)) { + fail("request.cache.retention", `${retention} is unsupported by this model`); + } + } + const sessionId = request.cache?.sessionId; + if (sessionId !== undefined) { + nonEmpty(sessionId, "request.cache.sessionId"); + if (retention === "none") + fail("request.cache.sessionId", "requires prompt caching to be enabled"); + if (Array.from(sessionId).length > 256) + fail("request.cache.sessionId", "must not exceed 256 characters"); + } + const placements: CachePlacement[] = []; + const compatibility = model.compatibility; + const usesContentMarkers = + retention !== "none" && + (model.apiDialect === "anthropic-messages" || + (compatibility?.dialect === "openai-chat" && + compatibility.cacheControlFormat === "anthropic") || + (compatibility?.dialect === "bedrock-converse-stream" && + compatibility.supportsPromptCacheMarkers === true)); + if (usesContentMarkers) { + if (request.system !== undefined && request.system.length > 0) + placements.push({ target: "system" }); + const supportsToolMarker = + compatibility?.dialect === "openai-chat" || + (compatibility?.dialect === "anthropic-messages" && + compatibility.supportsCacheControlOnTools === true); + if (supportsToolMarker && tools.length > 0) { + placements.push({ target: "tool", toolIndex: tools.length - 1 }); + } + let placedConversationMarker = false; + for ( + let messageIndex = messages.length - 1; + messageIndex >= 0 && !placedConversationMarker; + messageIndex -= 1 + ) { + const message = messages[messageIndex]; + if (message === undefined) continue; + for (let contentIndex = message.content.length - 1; contentIndex >= 0; contentIndex -= 1) { + const content = message.content[contentIndex]; + if (content?.type !== "text" && content?.type !== "blob") continue; + placements.push({ target: "message-content", messageIndex, contentIndex }); + placedConversationMarker = true; + break; + } + } + } + return Object.freeze({ + retention, + ...(sessionId === undefined || retention === "none" ? {} : { sessionId }), + placements: Object.freeze(placements), + }); +} + +async function loadBlobs( + model: ModelInfo, + messages: readonly RequestModelMessage[], + readBlob: ModelRequest["readBlob"], +): Promise> { + const references = new Map(); + for (const [messageIndex, message] of messages.entries()) { + for (const [contentIndex, content] of message.content.entries()) { + if (content.type !== "blob") continue; + if (!model.capabilities.imageInput) { + fail( + `request.messages[${messageIndex}].content[${contentIndex}]`, + "uses an image unsupported by this model", + ); + } + if (!content.blob.mediaType.startsWith("image/")) { + fail( + `request.messages[${messageIndex}].content[${contentIndex}].blob.mediaType`, + "must be an image media type", + ); + } + const existing = references.get(content.blob.sha256); + if ( + existing !== undefined && + (existing.mediaType !== content.blob.mediaType || + existing.sizeBytes !== content.blob.sizeBytes) + ) { + fail( + `request.messages[${messageIndex}].content[${contentIndex}].blob`, + "conflicts with another reference for the same digest", + ); + } + references.set(content.blob.sha256, content.blob); + } + } + if (references.size === 0) return new Map(); + if (readBlob === undefined) fail("request.readBlob", "is required when history contains blobs"); + const loaded = new Map(); + for (const [sha256, reference] of references) { + const bytes = await readBlob(reference); + if (!(bytes instanceof Uint8Array)) + fail(`request.blobs.${sha256}`, "loader must return Uint8Array"); + if (bytes.byteLength !== reference.sizeBytes) + fail(`request.blobs.${sha256}`, "size does not match its reference"); + const actualHash = createHash("sha256").update(bytes).digest("hex"); + if (actualHash !== sha256) + fail(`request.blobs.${sha256}`, "content hash does not match its reference"); + loaded.set( + sha256, + Object.freeze({ reference: Object.freeze({ ...reference }), bytes: bytes.slice() }), + ); + } + return loaded; +} + +function cloneBasicContent( + content: Extract["content"][number], + path: string, +) { + if (content.type === "text") { + exactKeys(content, ["type", "text"], path); + if (typeof content.text !== "string") fail(`${path}.text`, "must be a string"); + return Object.freeze({ type: "text" as const, text: content.text }); + } + if (content.type !== "blob") fail(`${path}.type`, "is not supported"); + exactKeys(content, ["type", "blob"], path); + exactKeys(content.blob, ["sha256", "mediaType", "sizeBytes", "name"], `${path}.blob`); + if (!/^[a-f0-9]{64}$/.test(content.blob.sha256)) + fail(`${path}.blob.sha256`, "must be a SHA-256 digest"); + nonEmpty(content.blob.mediaType, `${path}.blob.mediaType`); + if (!Number.isSafeInteger(content.blob.sizeBytes) || content.blob.sizeBytes < 0) { + fail(`${path}.blob.sizeBytes`, "must be a non-negative safe integer"); + } + if (content.blob.name !== undefined) nonEmpty(content.blob.name, `${path}.blob.name`); + return Object.freeze({ type: "blob" as const, blob: Object.freeze({ ...content.blob }) }); +} + +function sanitizeContent( + content: RequestAssistantContent, + target: ProviderModelIdentity, + path: string, + sanitizations: RequestSanitization[], +): RequestAssistantContent { + if (content.type === "text") { + exactKeys(content, ["type", "text", "signature", "continuation"], path); + if (typeof content.text !== "string") fail(`${path}.text`, "must be a string"); + const signature = keepSignature(content.signature, target, `${path}.signature`, sanitizations); + const continuation = keepContinuation( + content.continuation, + target, + `${path}.continuation`, + sanitizations, + ); + return Object.freeze({ + type: "text", + text: content.text, + ...(signature === undefined ? {} : { signature }), + ...(continuation === undefined ? {} : { continuation }), + }); + } + if (content.type === "thinking") { + exactKeys(content, ["type", "text", "signature", "redacted"], path); + if (typeof content.text !== "string") fail(`${path}.text`, "must be a string"); + if (content.redacted !== undefined && typeof content.redacted !== "boolean") { + fail(`${path}.redacted`, "must be a boolean"); + } + const signature = keepSignature(content.signature, target, `${path}.signature`, sanitizations); + if (content.redacted === true && signature === undefined) { + fail(path, "contains redacted reasoning that cannot be replayed by the selected model"); + } + if (content.text.length === 0 && signature === undefined) { + fail(path, "contains empty reasoning without a replayable signature"); + } + return Object.freeze({ + type: "thinking", + text: content.text, + ...(signature === undefined ? {} : { signature }), + ...(content.redacted === true ? { redacted: true } : {}), + }); + } + if (content.type !== "blob") fail(`${path}.type`, "is not supported"); + return cloneBasicContent(content, path); +} + +function prepareHistory( + model: ModelInfo, + messages: readonly RequestModelMessage[], + tools: readonly PreparedToolDeclaration[], + dialect: ToolDialectData, + sanitizations: RequestSanitization[], +): readonly PreparedRequestMessage[] { + const target = targetIdentity(model); + const activeTools = new Map(tools.map((tool) => [tool.canonicalName, tool])); + const callIds = new Map(); + const pending = new Map(); + const occupied = new Set(); + const output: PreparedRequestMessage[] = []; + + for (const [messageIndex, message] of messages.entries()) { + const path = `request.messages[${messageIndex}]`; + if (!Array.isArray(message.content) || message.content.length === 0) + fail(`${path}.content`, "must not be empty"); + if (message.role === "user") { + exactKeys(message, ["role", "content"], path); + if (pending.size > 0) fail(path, "appears before all preceding tool calls have results"); + output.push( + Object.freeze({ + role: "user", + content: Object.freeze( + message.content.map((item, contentIndex) => + cloneBasicContent(item, `${path}.content[${contentIndex}]`), + ), + ), + }), + ); + continue; + } + if (message.role === "assistant") { + exactKeys(message, ["role", "content", "toolCalls", "origin", "continuation"], path); + if (pending.size > 0) fail(path, "appears before all preceding tool calls have results"); + const origin = message.origin; + if (origin !== undefined) validateIdentity(origin, `${path}.origin`); + const continuation = keepContinuation( + message.continuation, + target, + `${path}.continuation`, + sanitizations, + ); + const content = Object.freeze( + message.content.map((item, contentIndex) => + sanitizeContent(item, target, `${path}.content[${contentIndex}]`, sanitizations), + ), + ); + const preparedCalls = message.toolCalls?.map((call, callIndex) => { + const callPath = `${path}.toolCalls[${callIndex}]`; + exactKeys(call, ["callId", "name", "input", "signature", "continuation"], callPath); + nonEmpty(call.callId, `${callPath}.callId`); + nonEmpty(call.name, `${callPath}.name`); + validateJson(call.input, `${callPath}.input`, new Set(), true); + if (callIds.has(call.callId)) + fail(`${callPath}.callId`, "is duplicated in request history"); + const normalized = normalizedToolCallId(call.callId, model.apiDialect, occupied); + callIds.set(call.callId, normalized); + occupied.add(normalized); + pending.set(call.callId, call.name); + const visibleName = activeTools.get(call.name)?.name ?? renderToolName(dialect, call.name); + const signature = keepSignature( + call.signature, + target, + `${callPath}.signature`, + sanitizations, + ); + const callContinuation = keepContinuation( + call.continuation, + target, + `${callPath}.continuation`, + sanitizations, + ); + return Object.freeze({ + callId: normalized, + canonicalCallId: call.callId, + name: visibleName, + canonicalName: call.name, + input: structuredClone(call.input), + ...(signature === undefined ? {} : { signature }), + ...(callContinuation === undefined ? {} : { continuation: callContinuation }), + }); + }); + output.push( + Object.freeze({ + role: "assistant", + content, + ...(preparedCalls === undefined ? {} : { toolCalls: Object.freeze(preparedCalls) }), + ...(origin === undefined ? {} : { origin: Object.freeze({ ...origin }) }), + ...(continuation === undefined ? {} : { continuation }), + }), + ); + continue; + } + if (message.role !== "tool") fail(`${path}.role`, "is not supported"); + exactKeys(message, ["role", "callId", "name", "content", "isError"], path); + nonEmpty(message.callId, `${path}.callId`); + nonEmpty(message.name, `${path}.name`); + if (typeof message.isError !== "boolean") fail(`${path}.isError`, "must be a boolean"); + const expectedName = pending.get(message.callId); + if (expectedName === undefined) + fail(`${path}.callId`, "does not match a preceding unresolved tool call"); + if (expectedName !== message.name) + fail(`${path}.name`, `does not match tool call ${expectedName}`); + const normalized = callIds.get(message.callId); + if (normalized === undefined) fail(`${path}.callId`, "has no normalized tool call identifier"); + pending.delete(message.callId); + output.push( + Object.freeze({ + role: "tool", + callId: normalized, + canonicalCallId: message.callId, + name: activeTools.get(message.name)?.name ?? renderToolName(dialect, message.name), + canonicalName: message.name, + content: Object.freeze( + message.content.map((item, contentIndex) => + cloneBasicContent(item, `${path}.content[${contentIndex}]`), + ), + ), + isError: message.isError, + }), + ); + } + if (pending.size > 0) fail("request.messages", "ends with unresolved tool calls"); + return Object.freeze(output); +} + +/** + * Validates and normalizes one canonical request before a native adapter renders it. + * The returned request contains no authentication material and records every removal + * of provider-bound replay metadata. + */ +export async function prepareModelRequest( + model: ModelInfo, + request: ModelRequest, + options: { readonly toolDialect?: ToolDialectData } = {}, +): Promise { + if (typeof request !== "object" || request === null || Array.isArray(request)) { + fail("request", "must be an object"); + } + exactKeys( + request, + [ + "modelId", + "system", + "messages", + "tools", + "thinkingLevel", + "thinkingBudgets", + "maxOutputTokens", + "httpIdleTimeoutMs", + "estimatedInputTokens", + "toolChoice", + "sampling", + "cache", + "safetySettings", + "metadata", + "readBlob", + "signal", + "timeoutMs", + "maxRetries", + "maxRetryDelayMs", + ], + "request", + ); + nonEmpty(request.modelId, "request.modelId"); + if (request.modelId !== model.modelId) fail("request.modelId", `does not match ${model.modelId}`); + if (request.system !== undefined && typeof request.system !== "string") { + fail("request.system", "must be a string"); + } + if (!Array.isArray(request.messages)) fail("request.messages", "must be an array"); + if (request.tools !== undefined && !Array.isArray(request.tools)) + fail("request.tools", "must be an array"); + if ( + request.toolChoice !== undefined && + request.toolChoice !== "auto" && + request.toolChoice !== "required" && + request.toolChoice !== "none" + ) { + fail("request.toolChoice", "is not recognized"); + } + if ( + request.thinkingLevel !== undefined && + !new Set(["off", "minimal", "low", "medium", "high", "xhigh", "max"]).has(request.thinkingLevel) + ) { + fail("request.thinkingLevel", "is not recognized"); + } + if (request.thinkingBudgets !== undefined) { + if ( + typeof request.thinkingBudgets !== "object" || + request.thinkingBudgets === null || + Array.isArray(request.thinkingBudgets) + ) { + fail("request.thinkingBudgets", "must be an object"); + } + exactKeys( + request.thinkingBudgets, + ["minimal", "low", "medium", "high"], + "request.thinkingBudgets", + ); + for (const [level, budget] of Object.entries(request.thinkingBudgets)) { + if (!Number.isSafeInteger(budget) || (budget as number) < 0) { + fail(`request.thinkingBudgets.${level}`, "must be a non-negative safe integer"); + } + } + } + for (const [field, value] of [ + ["timeoutMs", request.timeoutMs], + ["httpIdleTimeoutMs", request.httpIdleTimeoutMs], + ["estimatedInputTokens", request.estimatedInputTokens], + ["maxRetries", request.maxRetries], + ["maxRetryDelayMs", request.maxRetryDelayMs], + ] as const) { + if (value !== undefined && (!Number.isSafeInteger(value) || value < 0)) { + fail(`request.${field}`, "must be a non-negative safe integer"); + } + } + if (request.readBlob !== undefined && typeof request.readBlob !== "function") { + fail("request.readBlob", "must be a function"); + } + assertModelSupports(model, request); + validateMetadata(request.metadata); + const safetySettings = prepareSafetySettings(model, request.safetySettings); + const sampling = prepareSampling(model, request.sampling); + const sanitizations: RequestSanitization[] = []; + const dialect = options.toolDialect ?? toolDialectFor(model.apiDialect); + const tools = prepareTools(model, request.tools, dialect); + if (request.toolChoice === "required" && tools.length === 0) { + fail("request.toolChoice", "requires at least one tool"); + } + const messages = prepareHistory(model, request.messages, tools, dialect, sanitizations); + const blobs = await loadBlobs(model, messages, request.readBlob); + const reasoning = prepareReasoning(model, request); + if ( + sampling?.temperature !== undefined && + model.compatibility?.dialect === "anthropic-messages" && + model.compatibility.supportsTemperature !== true + ) { + fail("request.sampling.temperature", "is unsupported by this model"); + } + if ( + sampling?.temperature !== undefined && + model.apiDialect === "anthropic-messages" && + reasoning !== undefined && + reasoning.effective !== "off" + ) { + fail("request.sampling.temperature", "cannot be combined with Anthropic reasoning"); + } + const maxOutputTokens = resolvedMaxOutputTokens(model, request, reasoning); + const cache = prepareCache(model, request, tools, messages); + const readBlob = + blobs.size === 0 + ? request.readBlob + : async (reference: BlobReference): Promise => { + const blob = blobs.get(reference.sha256); + if (blob === undefined) fail(`request.blobs.${reference.sha256}`, "was not prepared"); + return blob.bytes.slice(); + }; + const preparation: RequestPreparation = Object.freeze({ + blobs, + tools, + ...(reasoning === undefined ? {} : { reasoning }), + cache, + sanitizations: Object.freeze(sanitizations), + }); + return Object.freeze({ + modelId: request.modelId, + ...(request.system === undefined ? {} : { system: request.system }), + messages, + ...(tools.length === 0 && request.tools === undefined ? {} : { tools }), + ...(reasoning === undefined ? {} : { thinkingLevel: reasoning.effective }), + ...(request.thinkingBudgets === undefined + ? {} + : { thinkingBudgets: Object.freeze({ ...request.thinkingBudgets }) }), + ...(maxOutputTokens === undefined ? {} : { maxOutputTokens }), + ...(request.httpIdleTimeoutMs === undefined + ? {} + : { httpIdleTimeoutMs: request.httpIdleTimeoutMs }), + ...(request.estimatedInputTokens === undefined + ? {} + : { estimatedInputTokens: request.estimatedInputTokens }), + ...(request.toolChoice === undefined ? {} : { toolChoice: request.toolChoice }), + ...(sampling === undefined ? {} : { sampling }), + cache, + ...(safetySettings === undefined ? {} : { safetySettings }), + ...(request.metadata === undefined ? {} : { metadata: Object.freeze({ ...request.metadata }) }), + ...(readBlob === undefined ? {} : { readBlob }), + ...(request.signal === undefined ? {} : { signal: request.signal }), + ...(request.timeoutMs === undefined ? {} : { timeoutMs: request.timeoutMs }), + ...(request.maxRetries === undefined ? {} : { maxRetries: request.maxRetries }), + ...(request.maxRetryDelayMs === undefined ? {} : { maxRetryDelayMs: request.maxRetryDelayMs }), + preparation, + }); +} + +export function preparedBlobDataUrl(blob: PreparedBlob): string { + return `data:${blob.reference.mediaType};base64,${Buffer.from(blob.bytes).toString("base64")}`; +} + +export function isPreparedModelRequest(request: ModelRequest): request is PreparedModelRequest { + return "preparation" in request; +} diff --git a/packages/ai/src/secret-context.ts b/packages/ai/src/secret-context.ts new file mode 100644 index 0000000..b10e65e --- /dev/null +++ b/packages/ai/src/secret-context.ts @@ -0,0 +1,17 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { AsyncLocalStorage } from "node:async_hooks"; + +const sinks = new AsyncLocalStorage<(values: readonly string[]) => void>(); + +export function withResolvedSecretSink( + sink: ((values: readonly string[]) => void) | undefined, + operation: () => Promise, +): Promise { + return sink === undefined ? operation() : sinks.run(sink, operation); +} + +export function registerResolvedSecrets(values: readonly string[]): void { + sinks.getStore()?.(values); +} diff --git a/packages/ai/src/sse.ts b/packages/ai/src/sse.ts index e33ae25..4c31150 100644 --- a/packages/ai/src/sse.ts +++ b/packages/ai/src/sse.ts @@ -1,6 +1,12 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan // SPDX-License-Identifier: Apache-2.0 +export const MAX_SSE_TOTAL_BYTES = 16 * 1024 * 1024; +export const MAX_SSE_LINE_BYTES = 1024 * 1024; +export const MAX_SSE_EVENT_NAME_BYTES = 256; +export const MAX_SSE_DATA_LINES = 1_024; +export const MAX_SSE_FRAME_DATA_BYTES = 4 * 1024 * 1024; + /** One server-sent event: optional event name plus joined data lines. */ export interface SseFrame { readonly event?: string; @@ -20,12 +26,17 @@ export async function* decodeSseStream( let buffer = ""; let eventName: string | undefined; let data: string[] = []; + let dataBytes = 0; + let totalBytes = 0; + const byteLength = (value: string): number => new TextEncoder().encode(value).byteLength; function* drainLines(): Generator { let newline = buffer.indexOf("\n"); while (newline !== -1) { let line = buffer.slice(0, newline); buffer = buffer.slice(newline + 1); + if (byteLength(line) > MAX_SSE_LINE_BYTES) + throw new TypeError("SSE line exceeds its byte limit"); if (line.endsWith("\r")) line = line.slice(0, -1); if (line === "") { @@ -34,20 +45,36 @@ export async function* decodeSseStream( } eventName = undefined; data = []; + dataBytes = 0; } else if (!line.startsWith(":")) { const colon = line.indexOf(":"); const field = colon === -1 ? line : line.slice(0, colon); let value = colon === -1 ? "" : line.slice(colon + 1); if (value.startsWith(" ")) value = value.slice(1); - if (field === "event") eventName = value; - else if (field === "data") data.push(value); + if (field === "event") { + if (byteLength(value) > MAX_SSE_EVENT_NAME_BYTES) + throw new TypeError("SSE event name exceeds its byte limit"); + eventName = value; + } else if (field === "data") { + if (data.length >= MAX_SSE_DATA_LINES) + throw new TypeError("SSE frame exceeds its data-line limit"); + dataBytes += byteLength(value) + (data.length === 0 ? 0 : 1); + if (dataBytes > MAX_SSE_FRAME_DATA_BYTES) + throw new TypeError("SSE frame data exceeds its byte limit"); + data.push(value); + } } newline = buffer.indexOf("\n"); } } for await (const chunk of source) { + totalBytes += chunk.byteLength; + if (totalBytes > MAX_SSE_TOTAL_BYTES) + throw new TypeError("SSE response exceeds its byte limit"); buffer += decoder.decode(chunk, { stream: true }); + if (!buffer.includes("\n") && byteLength(buffer) > MAX_SSE_LINE_BYTES) + throw new TypeError("SSE pending line exceeds its byte limit"); yield* drainLines(); } // End of source terminates any partial line and flushes the pending frame. diff --git a/packages/ai/src/static-openai-chat-provider.ts b/packages/ai/src/static-openai-chat-provider.ts new file mode 100644 index 0000000..4ab76f2 --- /dev/null +++ b/packages/ai/src/static-openai-chat-provider.ts @@ -0,0 +1,142 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { createEnvironmentApiKeyAuth } from "./api-key-auth.ts"; +import { + type AuthContext, + AuthError, + createProviderAuthentication, + type OAuthAuthMethod, + type ResolvedAuth, +} from "./auth.ts"; +import { getStaticModelCatalog } from "./catalog.ts"; +import { validateModelCatalog } from "./catalog-validation.ts"; +import { stripTrailingSlashes } from "./transport-safety.ts"; +import type { CredentialStore } from "./credentials.ts"; +import type { ModelInfo } from "./model.ts"; +import { type OpenAiChatEndpoint, OpenAiChatProvider } from "./openai-chat-provider.ts"; + +export interface StaticOpenAiChatProviderDefinition { + readonly id: string; + readonly displayName: string; + readonly apiKeyDisplayName: string; + readonly environmentVariables: readonly string[]; + readonly baseUrl: string; + readonly oauth?: OAuthAuthMethod; +} + +export interface StaticOpenAiChatProviderOptions { + readonly store: CredentialStore; + readonly context: AuthContext; + readonly models?: readonly ModelInfo[]; + readonly fetch?: typeof fetch; + readonly now?: () => number; +} + +function normalizedBaseUrl(value: string, providerId: string): string { + let url: URL; + try { + url = new URL(value); + } catch (cause) { + throw new TypeError(`Provider ${providerId} has an invalid fixed endpoint`, { cause }); + } + if ( + url.protocol !== "https:" || + url.username.length > 0 || + url.password.length > 0 || + url.search.length > 0 || + url.hash.length > 0 + ) { + throw new TypeError(`Provider ${providerId} has an unsafe fixed endpoint`); + } + return stripTrailingSlashes(url.toString()); +} + +function validateModels( + definition: StaticOpenAiChatProviderDefinition, + models: readonly ModelInfo[], +): readonly ModelInfo[] { + if (models.length === 0) { + throw new TypeError(`Provider ${definition.id} has no static models`); + } + const expectedBaseUrl = normalizedBaseUrl(definition.baseUrl, definition.id); + for (const model of models) { + if (model.providerId !== definition.id) { + throw new TypeError( + `Provider ${definition.id} cannot register model ${model.modelId} owned by ${model.providerId}`, + ); + } + if (model.apiDialect !== "openai-chat") { + throw new TypeError( + `Provider ${definition.id} model ${model.modelId} does not use openai-chat`, + ); + } + if ( + model.endpoint?.type !== "fixed" || + normalizedBaseUrl(model.endpoint.baseUrl, definition.id) !== expectedBaseUrl + ) { + throw new TypeError( + `Provider ${definition.id} model ${model.modelId} has an unexpected endpoint`, + ); + } + } + validateModelCatalog(models); + return models; +} + +/** Creates one static bearer-authenticated OpenAI Chat provider without side effects. */ +export function createStaticOpenAiChatProvider( + definition: StaticOpenAiChatProviderDefinition, + options: StaticOpenAiChatProviderOptions, +): OpenAiChatProvider { + const models = validateModels(definition, options.models ?? getStaticModelCatalog(definition.id)); + const apiKey = createEnvironmentApiKeyAuth({ + providerId: definition.id, + displayName: definition.apiKeyDisplayName, + environmentVariables: definition.environmentVariables, + }); + const authentication = createProviderAuthentication({ + providerId: definition.id, + declaredMethods: + definition.oauth === undefined ? ["environment", "file"] : ["environment", "file", "oauth"], + methods: { apiKey, ...(definition.oauth === undefined ? {} : { oauth: definition.oauth }) }, + store: options.store, + context: options.context, + }); + const expectedBaseUrl = normalizedBaseUrl(definition.baseUrl, definition.id); + const endpoint: OpenAiChatEndpoint = { + url: (model) => { + if ( + model.endpoint?.type !== "fixed" || + normalizedBaseUrl(model.endpoint.baseUrl, definition.id) !== expectedBaseUrl + ) { + throw new TypeError( + `Provider ${definition.id} model ${model.modelId} has an unexpected endpoint`, + ); + } + return `${expectedBaseUrl}/chat/completions`; + }, + headers: (model, resolved) => { + const key = resolved.auth.apiKey; + if (key === undefined || key.length === 0) { + throw new AuthError( + "invalid_auth", + definition.id, + `${definition.displayName} API key missing from resolved authentication`, + ); + } + return { ...model.headers, Authorization: `Bearer ${key}` }; + }, + }; + return new OpenAiChatProvider({ + id: definition.id, + displayName: definition.displayName, + authMethods: authentication.methods, + authentication, + endpoint, + models, + resolveAuth: (signal: AbortSignal): Promise => authentication.resolve({ signal }), + ...(options.fetch === undefined ? {} : { fetch: options.fetch }), + ...(options.now === undefined ? {} : { now: options.now }), + }); +} diff --git a/packages/ai/src/stream.ts b/packages/ai/src/stream.ts index 2111ec8..1f09acc 100644 --- a/packages/ai/src/stream.ts +++ b/packages/ai/src/stream.ts @@ -4,6 +4,7 @@ import { isTerminalModelStreamEvent, type ModelStreamEvent, + parseModelStreamEvent, type TerminalModelStreamEvent, } from "./model.ts"; @@ -18,16 +19,20 @@ export async function* normalizeModelStream( stream: AsyncIterable, signal?: AbortSignal, ): AsyncGenerator { + let emittedContent = false; try { - for await (const event of stream) { + for await (const rawEvent of stream) { + const event = parseModelStreamEvent(rawEvent); yield event; if (isTerminalModelStreamEvent(event)) return; + emittedContent = true; } } catch (error) { yield terminalForFailure( signal, "provider_stream_failure", error instanceof Error ? error.message : "provider stream threw a non-Error value", + emittedContent, ); return; } @@ -35,6 +40,7 @@ export async function* normalizeModelStream( signal, "provider_stream_truncated", "provider ended the stream without a terminal event", + emittedContent, ); } @@ -42,8 +48,9 @@ function terminalForFailure( signal: AbortSignal | undefined, code: string, message: string, + partial: boolean, ): TerminalModelStreamEvent { - if (signal?.aborted) return { type: "aborted" }; + if (signal?.aborted) return { type: "aborted", ...(partial ? { partial: true } : {}) }; return { type: "error", code, @@ -51,6 +58,7 @@ function terminalForFailure( retryable: false, category: "stream_interrupted", requestPhase: "streaming", + ...(partial ? { partial: true } : {}), }; } diff --git a/packages/ai/src/together.ts b/packages/ai/src/together.ts new file mode 100644 index 0000000..9f36a64 --- /dev/null +++ b/packages/ai/src/together.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const TOGETHER_PROVIDER_ID = "together"; +export const TOGETHER_API_KEY_ENV = "TOGETHER_API_KEY"; +export const TOGETHER_BASE_URL = "https://api.together.ai/v1"; + +export const TOGETHER_PROVIDER_DEFINITION = { + id: TOGETHER_PROVIDER_ID, + displayName: "Together AI", + apiKeyDisplayName: "Together API key", + environmentVariables: [TOGETHER_API_KEY_ENV], + baseUrl: TOGETHER_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createTogetherProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(TOGETHER_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/transport-safety.ts b/packages/ai/src/transport-safety.ts new file mode 100644 index 0000000..73df720 --- /dev/null +++ b/packages/ai/src/transport-safety.ts @@ -0,0 +1,338 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { lookup } from "node:dns/promises"; +import { isIP } from "node:net"; + +import { Agent, fetch as undiciFetch } from "undici"; + +export const MAX_ENDPOINT_LENGTH = 4_096; +export const MAX_JSON_RESPONSE_BYTES = 4 * 1024 * 1024; + +export function stripTrailingSlashes(value: string): string { + let end = value.length; + while (end > 0 && value.charCodeAt(end - 1) === 47) end -= 1; + return value.slice(0, end); +} + +function normalizedHostname(hostname: string): string { + return hostname + .replace(/^\[|\]$/g, "") + .replace(/\.$/, "") + .toLowerCase(); +} + +export function isLoopback(hostname: string): boolean { + const host = normalizedHostname(hostname); + if (host === "localhost" || host.endsWith(".localhost") || host === "::1") return true; + if (isIP(host) !== 4) return false; + return Number(host.split(".")[0]) === 127; +} + +export function isDisallowedIpv4(hostname: string): boolean { + if (isIP(hostname) !== 4) return false; + const octets = hostname.split(".").map(Number); + const [a = 0, b = 0, c = 0] = octets; + return ( + a === 0 || + a === 10 || + a === 127 || + (a === 100 && b >= 64 && b <= 127) || + (a === 169 && b === 254) || + (a === 172 && b >= 16 && b <= 31) || + (a === 192 && b === 0 && (c === 0 || c === 2)) || + (a === 192 && b === 88 && c === 99) || + (a === 192 && b === 168) || + (a === 198 && (b === 18 || b === 19 || (b === 51 && c === 100))) || + (a === 203 && b === 0 && c === 113) || + a >= 224 + ); +} + +export function isDisallowedIpv6(hostname: string): boolean { + const host = normalizedHostname(hostname); + if (isIP(host) !== 6) return false; + if (host === "::" || host === "::1" || host.startsWith("ff")) return true; + if (host.startsWith("::ffff:")) return true; + if (host.startsWith("fc") || host.startsWith("fd")) return true; + if (/^fe[89ab]/.test(host)) return true; + if (host.startsWith("2001:db8:")) return true; + const hextets = host.split(":"); + if (hextets[0] === "2001" && Number.parseInt(hextets[1] ?? "0", 16) < 0x200) return true; + return !/^[23][0-9a-f]{3}:/.test(host); +} + +export interface EndpointPolicyOptions { + readonly label: string; + readonly allowLoopbackHttp?: boolean; + readonly allowQuery?: boolean; + readonly expectedOrigin?: string; +} + +/** Validates and normalizes an endpoint before credentials or prompts can reach it. */ +export function safeEndpoint(value: string, options: EndpointPolicyOptions): string { + if (value.length === 0 || value.length > MAX_ENDPOINT_LENGTH) { + throw new TypeError(`${options.label} has an invalid length`); + } + let url: URL; + try { + url = new URL(value); + } catch { + throw new TypeError(`${options.label} is not a valid URL`); + } + if (url.username || url.password || (url.search && options.allowQuery !== true) || url.hash) { + throw new TypeError(`${options.label} must not contain credentials, a query, or a fragment`); + } + const loopback = isLoopback(url.hostname); + if (url.protocol !== "https:" && !(options.allowLoopbackHttp === true && loopback)) { + throw new TypeError(`${options.label} must use HTTPS, except for explicit loopback HTTP`); + } + const hostname = normalizedHostname(url.hostname); + if ( + (loopback && options.allowLoopbackHttp !== true) || + (!loopback && (isDisallowedIpv4(hostname) || isDisallowedIpv6(hostname))) || + hostname.endsWith(".local") + ) { + throw new TypeError(`${options.label} resolves to a disallowed destination`); + } + if (options.expectedOrigin !== undefined) { + const expected = safeEndpoint(options.expectedOrigin, { + label: `${options.label} source`, + ...(options.allowLoopbackHttp === undefined + ? {} + : { allowLoopbackHttp: options.allowLoopbackHttp }), + }); + if (url.origin !== new URL(expected).origin) { + throw new TypeError(`${options.label} changed to an unapproved origin`); + } + } + return stripTrailingSlashes(url.toString()); +} + +export interface SafeFetchOptions extends EndpointPolicyOptions { + readonly fetch?: typeof fetch; + readonly maximumRedirects?: number; + readonly idleTimeoutMs?: number; + readonly resolve?: (hostname: string) => Promise; +} + +function allowedAddress(address: string, allowLoopbackHttp: boolean): boolean { + if (isLoopback(address)) return allowLoopbackHttp; + return !isDisallowedIpv4(address) && !isDisallowedIpv6(address); +} + +async function validatedAddresses( + url: URL, + options: SafeFetchOptions, +): Promise { + const literalFamily = isIP(url.hostname.replace(/^\[|\]$/g, "")); + const addresses = + literalFamily === 0 + ? options.fetch !== undefined && options.resolve === undefined + ? [{ address: "8.8.8.8", family: 4 as const }] + : await ( + options.resolve ?? + (async (hostname) => + (await lookup(hostname, { all: true, verbatim: true })).map((entry) => ({ + address: entry.address, + family: entry.family as 4 | 6, + }))) + )(url.hostname) + : [{ address: url.hostname.replace(/^\[|\]$/g, ""), family: literalFamily as 4 | 6 }]; + if ( + addresses.length === 0 || + addresses.some(({ address }) => !allowedAddress(address, options.allowLoopbackHttp === true)) + ) { + throw new TypeError(`${options.label} resolves to a disallowed destination`); + } + return addresses; +} + +function responseWithAgent(response: Response, agent: Agent): Response { + if (response.body === null) { + void agent.close(); + return response; + } + const reader = response.body.getReader(); + const body = new ReadableStream({ + async pull(controller) { + try { + const result = await reader.read(); + if (result.done) { + controller.close(); + await agent.close(); + } else controller.enqueue(result.value); + } catch (error) { + controller.error(error); + await agent.close(); + } + }, + async cancel(reason) { + try { + await reader.cancel(reason); + } finally { + await agent.close(); + } + }, + }); + return new Response(body, { + status: response.status, + statusText: response.statusText, + headers: response.headers, + }); +} + +/** Resolves and pins each destination, and validates every bounded redirect before dispatch. */ +export async function safeFetch( + input: string | URL, + init: RequestInit = {}, + options: SafeFetchOptions, +): Promise { + const maximumRedirects = options.maximumRedirects ?? 3; + let url = new URL( + safeEndpoint(String(input), { + label: options.label, + ...(options.allowLoopbackHttp === undefined + ? {} + : { allowLoopbackHttp: options.allowLoopbackHttp }), + allowQuery: true, + ...(options.expectedOrigin === undefined ? {} : { expectedOrigin: options.expectedOrigin }), + }), + ); + const approvedOrigin = options.expectedOrigin ?? url.origin; + for (let redirects = 0; ; redirects += 1) { + if (redirects > maximumRedirects) + throw new TypeError(`${options.label} redirected too many times`); + const addresses = await validatedAddresses(url, options); + const pinned = addresses[0]; + if (pinned === undefined) throw new TypeError(`${options.label} has no approved destination`); + const requestInit = { ...init, redirect: "manual" as const }; + let response: Response; + if (options.fetch !== undefined) { + response = await options.fetch(url, requestInit); + } else { + const agent = new Agent({ + ...(options.idleTimeoutMs === undefined + ? {} + : { headersTimeout: options.idleTimeoutMs, bodyTimeout: options.idleTimeoutMs }), + connect: { + lookup: (_hostname, lookupOptions, callback) => + lookupOptions.all + ? callback(null, [pinned]) + : callback(null, pinned.address, pinned.family), + }, + }); + try { + response = responseWithAgent( + (await undiciFetch(url, { + ...requestInit, + dispatcher: agent, + } as unknown as Parameters[1])) as unknown as Response, + agent, + ); + } catch (error) { + await agent.close(); + throw error; + } + } + if (![301, 302, 303, 307, 308].includes(response.status)) return response; + const location = response.headers.get("location"); + await response.body?.cancel(); + if (location === null) + throw new TypeError(`${options.label} returned a redirect without a location`); + url = new URL( + safeEndpoint(new URL(location, url).toString(), { + label: `${options.label} redirect`, + ...(options.allowLoopbackHttp === undefined + ? {} + : { allowLoopbackHttp: options.allowLoopbackHttp }), + allowQuery: true, + expectedOrigin: approvedOrigin, + }), + ); + } +} + +export function delayWithSignal(milliseconds: number, signal: AbortSignal): Promise { + signal.throwIfAborted(); + return new Promise((resolve, reject) => { + const timer = setTimeout(() => { + signal.removeEventListener("abort", abort); + resolve(); + }, milliseconds); + const abort = () => { + clearTimeout(timer); + reject(signal.reason); + }; + signal.addEventListener("abort", abort, { once: true }); + }); +} + +export function raceWithSignal( + operation: Promise, + signal: AbortSignal, +): Promise { + signal.throwIfAborted(); + return new Promise((resolve, reject) => { + const abort = () => reject(signal.reason); + signal.addEventListener("abort", abort, { once: true }); + operation.then( + (value) => { + signal.removeEventListener("abort", abort); + resolve(value); + }, + (error: unknown) => { + signal.removeEventListener("abort", abort); + reject(error); + }, + ); + }); +} + +export async function readBoundedBody( + response: Pick, + maximumBytes = MAX_JSON_RESPONSE_BYTES, + signal?: AbortSignal, +): Promise { + const declared = Number(response.headers.get("content-length")); + if (Number.isFinite(declared) && declared > maximumBytes) { + throw new TypeError(`Provider response exceeds ${maximumBytes} bytes`); + } + if (response.body === null) return new Uint8Array(); + const chunks: Uint8Array[] = []; + let length = 0; + const reader = response.body.getReader(); + try { + for (;;) { + const result = + signal === undefined ? await reader.read() : await raceWithSignal(reader.read(), signal); + if (result.done) break; + length += result.value.byteLength; + if (length > maximumBytes) + throw new TypeError(`Provider response exceeds ${maximumBytes} bytes`); + chunks.push(result.value); + } + } finally { + reader.releaseLock(); + } + const body = new Uint8Array(length); + let offset = 0; + for (const chunk of chunks) { + body.set(chunk, offset); + offset += chunk.byteLength; + } + return body; +} + +export async function readBoundedJson( + response: Pick, + maximumBytes = MAX_JSON_RESPONSE_BYTES, + signal?: AbortSignal, +): Promise { + const bytes = await readBoundedBody(response, maximumBytes, signal); + try { + return JSON.parse(new TextDecoder("utf-8", { fatal: true }).decode(bytes)) as unknown; + } catch (cause) { + throw new TypeError("Provider response is not valid bounded JSON", { cause }); + } +} diff --git a/packages/ai/src/usage.ts b/packages/ai/src/usage.ts index 1cc9401..8e7d481 100644 --- a/packages/ai/src/usage.ts +++ b/packages/ai/src/usage.ts @@ -3,7 +3,7 @@ import type { Usage } from "@axl/protocol"; -import type { ModelCost } from "./model.ts"; +import type { ModelCost, ModelCostRates } from "./model.ts"; export function emptyUsage(): Usage { return { @@ -28,16 +28,32 @@ export function addUsage(total: Usage, delta: Usage): Usage { }; } +/** Selects the highest request-wide price tier matching total input usage. */ +export function modelCostRates(cost: ModelCost, usage: Usage): ModelCostRates { + const totalInput = usage.inputTokens + usage.cacheReadTokens + usage.cacheWriteTokens; + let selected: ModelCostRates = cost; + let selectedThreshold = -1; + for (const tier of cost.tiers ?? []) { + if (tier.inputTokensAbove < totalInput && tier.inputTokensAbove > selectedThreshold) { + selected = tier; + selectedThreshold = tier.inputTokensAbove; + } + } + return selected; +} + /** - * Cost of one usage under a model's rates. Cache rates a provider does not - * publish count as zero; providers with cache pricing must supply them. + * Cost of one usage under the matching request-wide rates. Cache rates a + * provider does not publish count as zero; providers with cache pricing must + * supply them. */ export function usageCostUsd(cost: ModelCost, usage: Usage): number { + const rates = modelCostRates(cost, usage); return ( - (cost.inputUsdPerMTok * usage.inputTokens + - cost.outputUsdPerMTok * usage.outputTokens + - (cost.cacheReadUsdPerMTok ?? 0) * usage.cacheReadTokens + - (cost.cacheWriteUsdPerMTok ?? 0) * usage.cacheWriteTokens) / + (rates.inputUsdPerMTok * usage.inputTokens + + rates.outputUsdPerMTok * usage.outputTokens + + (rates.cacheReadUsdPerMTok ?? 0) * usage.cacheReadTokens + + (rates.cacheWriteUsdPerMTok ?? 0) * usage.cacheWriteTokens) / 1_000_000 ); } diff --git a/packages/ai/src/vercel-ai-gateway.ts b/packages/ai/src/vercel-ai-gateway.ts new file mode 100644 index 0000000..6e48f64 --- /dev/null +++ b/packages/ai/src/vercel-ai-gateway.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const VERCEL_AI_GATEWAY_PROVIDER_ID = "vercel-ai-gateway"; +export const VERCEL_AI_GATEWAY_API_KEY_ENV = "AI_GATEWAY_API_KEY"; +export const VERCEL_AI_GATEWAY_BASE_URL = "https://ai-gateway.vercel.sh/v1"; + +export const VERCEL_AI_GATEWAY_PROVIDER_DEFINITION = { + id: VERCEL_AI_GATEWAY_PROVIDER_ID, + displayName: "Vercel AI Gateway", + apiKeyDisplayName: "Vercel AI Gateway API key", + environmentVariables: [VERCEL_AI_GATEWAY_API_KEY_ENV], + baseUrl: VERCEL_AI_GATEWAY_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createVercelAiGatewayProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(VERCEL_AI_GATEWAY_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/xai.ts b/packages/ai/src/xai.ts new file mode 100644 index 0000000..0654ff9 --- /dev/null +++ b/packages/ai/src/xai.ts @@ -0,0 +1,28 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; +import { createXaiOAuth } from "./oauth-auth.ts"; + +export const XAI_PROVIDER_ID = "xai"; +export const XAI_API_KEY_ENV = "XAI_API_KEY"; +export const XAI_BASE_URL = "https://api.x.ai/v1"; + +export const XAI_PROVIDER_DEFINITION = { + id: XAI_PROVIDER_ID, + displayName: "xAI", + apiKeyDisplayName: "xAI API key", + environmentVariables: [XAI_API_KEY_ENV], + baseUrl: XAI_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createXaiProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider( + { ...XAI_PROVIDER_DEFINITION, oauth: createXaiOAuth(options) }, + options, + ); +} diff --git a/packages/ai/src/xiaomi-token-plan-ams.ts b/packages/ai/src/xiaomi-token-plan-ams.ts new file mode 100644 index 0000000..b3f07e2 --- /dev/null +++ b/packages/ai/src/xiaomi-token-plan-ams.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const XIAOMI_TOKEN_PLAN_AMS_PROVIDER_ID = "xiaomi-token-plan-ams"; +export const XIAOMI_TOKEN_PLAN_AMS_API_KEY_ENV = "XIAOMI_TOKEN_PLAN_AMS_API_KEY"; +export const XIAOMI_TOKEN_PLAN_AMS_BASE_URL = "https://token-plan-ams.xiaomimimo.com/v1"; + +export const XIAOMI_TOKEN_PLAN_AMS_PROVIDER_DEFINITION = { + id: XIAOMI_TOKEN_PLAN_AMS_PROVIDER_ID, + displayName: "Xiaomi Token Plan Amsterdam", + apiKeyDisplayName: "Xiaomi Token Plan Amsterdam API key", + environmentVariables: [XIAOMI_TOKEN_PLAN_AMS_API_KEY_ENV], + baseUrl: XIAOMI_TOKEN_PLAN_AMS_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createXiaomiTokenPlanAmsProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(XIAOMI_TOKEN_PLAN_AMS_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/xiaomi-token-plan-cn.ts b/packages/ai/src/xiaomi-token-plan-cn.ts new file mode 100644 index 0000000..a0580c6 --- /dev/null +++ b/packages/ai/src/xiaomi-token-plan-cn.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const XIAOMI_TOKEN_PLAN_CN_PROVIDER_ID = "xiaomi-token-plan-cn"; +export const XIAOMI_TOKEN_PLAN_CN_API_KEY_ENV = "XIAOMI_TOKEN_PLAN_CN_API_KEY"; +export const XIAOMI_TOKEN_PLAN_CN_BASE_URL = "https://token-plan-cn.xiaomimimo.com/v1"; + +export const XIAOMI_TOKEN_PLAN_CN_PROVIDER_DEFINITION = { + id: XIAOMI_TOKEN_PLAN_CN_PROVIDER_ID, + displayName: "Xiaomi Token Plan China", + apiKeyDisplayName: "Xiaomi Token Plan China API key", + environmentVariables: [XIAOMI_TOKEN_PLAN_CN_API_KEY_ENV], + baseUrl: XIAOMI_TOKEN_PLAN_CN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createXiaomiTokenPlanCnProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(XIAOMI_TOKEN_PLAN_CN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/xiaomi-token-plan-sgp.ts b/packages/ai/src/xiaomi-token-plan-sgp.ts new file mode 100644 index 0000000..7115bf8 --- /dev/null +++ b/packages/ai/src/xiaomi-token-plan-sgp.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const XIAOMI_TOKEN_PLAN_SGP_PROVIDER_ID = "xiaomi-token-plan-sgp"; +export const XIAOMI_TOKEN_PLAN_SGP_API_KEY_ENV = "XIAOMI_TOKEN_PLAN_SGP_API_KEY"; +export const XIAOMI_TOKEN_PLAN_SGP_BASE_URL = "https://token-plan-sgp.xiaomimimo.com/v1"; + +export const XIAOMI_TOKEN_PLAN_SGP_PROVIDER_DEFINITION = { + id: XIAOMI_TOKEN_PLAN_SGP_PROVIDER_ID, + displayName: "Xiaomi Token Plan Singapore", + apiKeyDisplayName: "Xiaomi Token Plan Singapore API key", + environmentVariables: [XIAOMI_TOKEN_PLAN_SGP_API_KEY_ENV], + baseUrl: XIAOMI_TOKEN_PLAN_SGP_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createXiaomiTokenPlanSgpProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(XIAOMI_TOKEN_PLAN_SGP_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/xiaomi.ts b/packages/ai/src/xiaomi.ts new file mode 100644 index 0000000..c1814df --- /dev/null +++ b/packages/ai/src/xiaomi.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const XIAOMI_PROVIDER_ID = "xiaomi"; +export const XIAOMI_API_KEY_ENV = "XIAOMI_API_KEY"; +export const XIAOMI_BASE_URL = "https://api.xiaomimimo.com/v1"; + +export const XIAOMI_PROVIDER_DEFINITION = { + id: XIAOMI_PROVIDER_ID, + displayName: "Xiaomi MiMo", + apiKeyDisplayName: "Xiaomi API key", + environmentVariables: [XIAOMI_API_KEY_ENV], + baseUrl: XIAOMI_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createXiaomiProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(XIAOMI_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/zai-coding-cn.ts b/packages/ai/src/zai-coding-cn.ts new file mode 100644 index 0000000..2218fa0 --- /dev/null +++ b/packages/ai/src/zai-coding-cn.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const ZAI_CODING_CN_PROVIDER_ID = "zai-coding-cn"; +export const ZAI_CODING_CN_API_KEY_ENV = "ZAI_CODING_CN_API_KEY"; +export const ZAI_CODING_CN_BASE_URL = "https://open.bigmodel.cn/api/coding/paas/v4"; + +export const ZAI_CODING_CN_PROVIDER_DEFINITION = { + id: ZAI_CODING_CN_PROVIDER_ID, + displayName: "Z.AI Coding China", + apiKeyDisplayName: "Z.AI Coding China API key", + environmentVariables: [ZAI_CODING_CN_API_KEY_ENV], + baseUrl: ZAI_CODING_CN_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createZaiCodingCnProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(ZAI_CODING_CN_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/src/zai.ts b/packages/ai/src/zai.ts new file mode 100644 index 0000000..c79469c --- /dev/null +++ b/packages/ai/src/zai.ts @@ -0,0 +1,24 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { + createStaticOpenAiChatProvider, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, +} from "./static-openai-chat-provider.ts"; + +export const ZAI_PROVIDER_ID = "zai"; +export const ZAI_API_KEY_ENV = "ZAI_API_KEY"; +export const ZAI_BASE_URL = "https://api.z.ai/api/paas/v4"; + +export const ZAI_PROVIDER_DEFINITION = { + id: ZAI_PROVIDER_ID, + displayName: "Z.AI", + apiKeyDisplayName: "Z.AI API key", + environmentVariables: [ZAI_API_KEY_ENV], + baseUrl: ZAI_BASE_URL, +} as const satisfies StaticOpenAiChatProviderDefinition; + +export function createZaiProvider(options: StaticOpenAiChatProviderOptions) { + return createStaticOpenAiChatProvider(ZAI_PROVIDER_DEFINITION, options); +} diff --git a/packages/ai/test/anthropic-messages.test.ts b/packages/ai/test/anthropic-messages.test.ts new file mode 100644 index 0000000..7993d09 --- /dev/null +++ b/packages/ai/test/anthropic-messages.test.ts @@ -0,0 +1,597 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + AnthropicMessagesCodecError, + decodeAnthropicMessagesStream, + encodeAnthropicMessagesRequest, + getStaticModelCatalog, + type ModelInfo, + type ModelRequest, + normalizeModelStream, + prepareModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function anthropicModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "anthropic", + modelId: "claude-fixture", + displayName: "Claude Fixture", + apiDialect: "anthropic-messages", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { minimal: "low", low: "low", medium: "medium", high: "high" }, + contextWindow: 200_000, + maxOutputTokens: 32_000, + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.1, + cacheWriteUsdPerMTok: 1.25, + }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short", "long"], + }, + sampling: { + supported: ["temperature", "topP", "topK"], + customFields: ["stop_sequences"], + }, + compatibility: { + dialect: "anthropic-messages", + supportsLongCacheRetention: true, + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + forceAdaptiveThinking: true, + }, + ...overrides, + }; +} + +const baseRequest: ModelRequest = { + modelId: "claude-fixture", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], +}; + +async function prepared(request: ModelRequest = baseRequest, model = anthropicModel()) { + return prepareModelRequest(model, request); +} + +async function* frames(events: readonly unknown[]): AsyncGenerator { + for (const event of events) { + const value = event as { type?: string }; + yield { + ...(value.type === undefined ? {} : { event: value.type }), + data: JSON.stringify(event), + }; + } +} + +async function decode(events: readonly unknown[], request?: Awaited>) { + return Array.fromAsync( + decodeAnthropicMessagesStream(frames(events), { + model: anthropicModel(), + request: request ?? (await prepared()), + }), + ); +} + +function terminalEvents(reason: string, details?: Record): unknown[] { + return [ + { + type: "message_start", + message: { + id: "msg-fixture", + model: "claude-routed", + usage: { + input_tokens: 10, + output_tokens: 0, + cache_read_input_tokens: 4, + cache_creation_input_tokens: 6, + cache_creation: { ephemeral_1h_input_tokens: 2 }, + }, + }, + }, + { + type: "message_delta", + delta: { stop_reason: reason, ...(details === undefined ? {} : { stop_details: details }) }, + usage: { output_tokens: 8, output_tokens_details: { thinking_tokens: 3 } }, + }, + { type: "message_stop" }, + ]; +} + +test("marks current adaptive Anthropic models explicitly in the generated catalog", () => { + const adaptive = getStaticModelCatalog("anthropic") + .filter( + (model) => + model.compatibility?.dialect === "anthropic-messages" && + model.compatibility.forceAdaptiveThinking === true, + ) + .map((model) => model.modelId) + .sort(); + assert.deepEqual(adaptive, [ + "claude-fable-5", + "claude-fable-5-1", + "claude-opus-4-8", + "claude-opus-5", + "claude-sonnet-5", + ]); +}); + +test("encodes prepared Anthropic history, images, thinking, tools, cache, and controls", async () => { + const model = anthropicModel(); + const bytes = new Uint8Array([1, 2, 3, 4]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const request: ModelRequest = { + modelId: model.modelId, + system: "Use evidence.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: bytes.length } }, + ], + }, + { + role: "assistant", + content: [ + { type: "thinking", text: "checked", signature: { ...identity, value: "signed-1" } }, + { + type: "thinking", + text: "", + signature: { ...identity, value: "redacted-1" }, + redacted: true, + }, + { type: "text", text: "calling" }, + ], + toolCalls: [{ callId: "call-1", name: "lookup", input: { key: "a" } }], + }, + { + role: "tool", + callId: "call-1", + name: "lookup", + content: [{ type: "text", text: "value" }], + isError: false, + }, + { role: "user", content: [{ type: "text", text: "continue" }] }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { key: { type: "string" }, optional: { type: "number" } }, + required: ["key"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "medium", + maxOutputTokens: 40, + toolChoice: "required", + sampling: { topP: 0.8, topK: 20, custom: { stop_sequences: ["END"] } }, + cache: { retention: "long" }, + metadata: { user_id: "user-fixture" }, + readBlob: async () => bytes, + }; + assert.throws( + () => encodeAnthropicMessagesRequest(model, request as never), + /requires a prepared model request/, + ); + + const encoded = encodeAnthropicMessagesRequest(model, await prepared(request, model)); + assert.deepEqual(encoded.headers, { + accept: "text/event-stream", + "content-type": "application/json", + "anthropic-version": "2023-06-01", + }); + assert.deepEqual(encoded.body, { + model: "claude-fixture", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { + type: "image", + source: { type: "base64", media_type: "image/png", data: "AQIDBA==" }, + }, + ], + }, + { + role: "assistant", + content: [ + { type: "thinking", thinking: "checked", signature: "signed-1" }, + { type: "redacted_thinking", data: "redacted-1" }, + { type: "text", text: "calling" }, + { type: "tool_use", id: "call-1", name: "lookup", input: { key: "a" } }, + ], + }, + { + role: "user", + content: [ + { + type: "tool_result", + tool_use_id: "call-1", + content: [{ type: "text", text: "value" }], + is_error: false, + }, + ], + }, + { + role: "user", + content: [ + { + type: "text", + text: "continue", + cache_control: { type: "ephemeral", ttl: "1h" }, + }, + ], + }, + ], + max_tokens: 40, + stream: true, + system: [ + { + type: "text", + text: "Use evidence.", + cache_control: { type: "ephemeral", ttl: "1h" }, + }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + input_schema: { + type: "object", + properties: { + key: { type: "string" }, + optional: { anyOf: [{ type: "number" }, { type: "null" }] }, + }, + required: ["key", "optional"], + additionalProperties: false, + }, + strict: true, + cache_control: { type: "ephemeral", ttl: "1h" }, + }, + ], + tool_choice: { type: "any" }, + thinking: { type: "adaptive", display: "summarized" }, + output_config: { effort: "medium" }, + top_p: 0.8, + top_k: 20, + stop_sequences: ["END"], + metadata: { user_id: "user-fixture" }, + }); +}); + +test("uses prepared budgets for legacy thinking and supports explicit thinking disable", async () => { + const model = anthropicModel({ + thinkingLevelMap: {}, + compatibility: { + dialect: "anthropic-messages", + supportsLongCacheRetention: true, + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + }, + }); + const enabledRequest = await prepared( + { ...baseRequest, thinkingLevel: "medium", maxOutputTokens: 100, cache: { retention: "none" } }, + model, + ); + assert.equal(enabledRequest.maxOutputTokens, 8_292); + assert.deepEqual(encodeAnthropicMessagesRequest(model, enabledRequest), { + body: { + model: "claude-fixture", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + max_tokens: 8_292, + stream: true, + thinking: { type: "enabled", budget_tokens: 7_268, display: "summarized" }, + }, + headers: { + accept: "text/event-stream", + "content-type": "application/json", + "anthropic-version": "2023-06-01", + "anthropic-beta": "interleaved-thinking-2025-05-14", + }, + }); + + const disabled = await prepared( + { + ...baseRequest, + thinkingLevel: "off", + sampling: { temperature: 0 }, + cache: { retention: "none" }, + }, + model, + ); + const disabledBody = encodeAnthropicMessagesRequest(model, disabled).body; + assert.deepEqual(disabledBody.thinking, { type: "disabled" }); + assert.equal(disabledBody.temperature, 0); +}); + +test("fails unsupported Anthropic request behavior without compatibility fallback", async () => { + const model = anthropicModel({ + cache: { supported: true, defaultRetention: "short", supportedRetentions: ["none", "short"] }, + compatibility: { + dialect: "anthropic-messages", + supportsLongCacheRetention: false, + supportsCacheControlOnTools: true, + supportsTemperature: true, + supportsStrictTools: true, + forceAdaptiveThinking: true, + }, + }); + await assert.rejects( + prepareModelRequest(model, { ...baseRequest, cache: { retention: "long" } }), + /long is unsupported/, + ); + await assert.rejects( + prepareModelRequest(model, { + ...baseRequest, + thinkingLevel: "high", + sampling: { temperature: 0 }, + }), + /cannot be combined with Anthropic reasoning/, + ); + const metadata = await prepared({ ...baseRequest, metadata: { trace: "safe" } }); + assert.throws( + () => encodeAnthropicMessagesRequest(anthropicModel(), metadata), + /supports only metadata.user_id/, + ); +}); + +test("decodes interleaved thinking, redaction, text, tools, usage, and provenance", async () => { + const events = await decode([ + terminalEvents("tool_use")[0], + { + type: "content_block_start", + index: 0, + content_block: { type: "thinking", thinking: "start", signature: "sig-" }, + }, + { type: "content_block_delta", index: 0, delta: { type: "thinking_delta", thinking: "end" } }, + { type: "content_block_delta", index: 0, delta: { type: "signature_delta", signature: "one" } }, + { type: "content_block_stop", index: 0 }, + { + type: "content_block_start", + index: 1, + content_block: { type: "redacted_thinking", data: "redacted-sig" }, + }, + { type: "content_block_stop", index: 1 }, + { type: "content_block_start", index: 2, content_block: { type: "text", text: "hello " } }, + { type: "content_block_delta", index: 2, delta: { type: "text_delta", text: "world" } }, + { type: "content_block_stop", index: 2 }, + { + type: "content_block_start", + index: 3, + content_block: { type: "tool_use", id: "tool-1", name: "lookup", input: {} }, + }, + { + type: "content_block_delta", + index: 3, + delta: { type: "input_json_delta", partial_json: '{"key":"a"}' }, + }, + { type: "content_block_stop", index: 3 }, + terminalEvents("tool_use")[1], + terminalEvents("tool_use")[2], + { type: "future_event", ignored: true }, + ]); + + assert.deepEqual( + events.map((event) => event.type), + [ + "thinking_delta", + "thinking_delta", + "replay_metadata", + "thinking_delta", + "replay_metadata", + "text_delta", + "text_delta", + "tool_call_start", + "tool_call_delta", + "tool_call", + "completed", + ], + ); + assert.deepEqual(events[2], { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "anthropic", + apiDialect: "anthropic-messages", + modelId: "claude-fixture", + signature: "sig-one", + }); + assert.deepEqual(events[4], { + type: "replay_metadata", + target: "thinking", + contentIndex: 1, + providerId: "anthropic", + apiDialect: "anthropic-messages", + modelId: "claude-fixture", + signature: "redacted-sig", + redacted: true, + }); + assert.deepEqual(events[9], { + type: "tool_call", + contentIndex: 3, + callId: "tool-1", + name: "lookup", + input: { key: "a" }, + }); + assert.deepEqual(events[10], { + type: "completed", + stopReason: "tool_use", + usage: { + inputTokens: 10, + outputTokens: 8, + cacheReadTokens: 4, + cacheWriteTokens: 6, + reasoningTokens: 3, + costUsd: 0.0000354, + }, + response: { + providerId: "anthropic", + requestedModelId: "claude-fixture", + routedModelId: "claude-routed", + responseId: "msg-fixture", + nativeStopReason: "tool_use", + }, + }); +}); + +test("maps native completion limits and refusal details", async () => { + const stopped = await decode(terminalEvents("end_turn")); + const stoppedTerminal = stopped.at(-1); + assert.equal(stoppedTerminal?.type, "completed"); + if (stoppedTerminal?.type === "completed") assert.equal(stoppedTerminal.stopReason, "stop"); + + const limited = await decode(terminalEvents("max_tokens")); + assert.deepEqual(limited.at(-1), { + type: "completed", + stopReason: "length", + partial: true, + usage: { + inputTokens: 10, + outputTokens: 8, + cacheReadTokens: 4, + cacheWriteTokens: 6, + reasoningTokens: 3, + costUsd: 0.0000354, + }, + response: { + providerId: "anthropic", + requestedModelId: "claude-fixture", + routedModelId: "claude-routed", + responseId: "msg-fixture", + nativeStopReason: "max_tokens", + }, + }); + + const refused = await decode(terminalEvents("refusal", { explanation: "request rejected" })); + assert.deepEqual(refused.at(-1), { + type: "error", + code: "refusal", + message: "request rejected", + retryable: false, + response: { + providerId: "anthropic", + requestedModelId: "claude-fixture", + routedModelId: "claude-routed", + responseId: "msg-fixture", + nativeStopReason: "refusal", + }, + }); +}); + +test("redacts provider errors and reports cancellation after partial output", async () => { + const secret = "credential-fixture-value"; + const failed = await Array.fromAsync( + decodeAnthropicMessagesStream( + frames([ + { type: "content_block_start", index: 0, content_block: { type: "text", text: "safe" } }, + { type: "content_block_stop", index: 0 }, + { type: "error", error: { type: "overloaded_error", message: `${secret} overloaded` } }, + ]), + { model: anthropicModel(), request: await prepared(), secretValues: [secret] }, + ), + ); + const terminal = failed.at(-1); + assert.equal(terminal?.type, "error"); + if (terminal?.type === "error") { + assert.equal(terminal.retryable, true); + assert.equal(terminal.partial, true); + assert.equal(terminal.message.includes(secret), false); + } + + const controller = new AbortController(); + const request = await prepared({ ...baseRequest, signal: controller.signal }); + async function* cancellingFrames(): AsyncGenerator { + yield { + event: "content_block_start", + data: JSON.stringify({ + type: "content_block_start", + index: 0, + content_block: { type: "text", text: "partial" }, + }), + }; + controller.abort(); + yield { event: "message_stop", data: JSON.stringify({ type: "message_stop" }) }; + } + const aborted = await Array.fromAsync( + decodeAnthropicMessagesStream(cancellingFrames(), { model: anthropicModel(), request }), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); +}); + +test("fails malformed input and normalizes truncation exactly once while ignoring unknown events", async () => { + await assert.rejects( + decode([ + { + type: "content_block_start", + index: 0, + content_block: { type: "tool_use", id: "tool-1", name: "lookup", input: {} }, + }, + { + type: "content_block_delta", + index: 0, + delta: { type: "input_json_delta", partial_json: "{bad" }, + }, + { type: "content_block_stop", index: 0 }, + ]), + AnthropicMessagesCodecError, + ); + + const malformed = await Array.fromAsync( + normalizeModelStream( + decodeAnthropicMessagesStream( + (async function* () { + yield { event: "message_start", data: "{bad json" }; + })(), + { model: anthropicModel(), request: await prepared() }, + ), + ), + ); + assert.equal(malformed.length, 1); + assert.equal(malformed[0]?.type, "error"); + + const truncated = await Array.fromAsync( + normalizeModelStream( + decodeAnthropicMessagesStream( + frames([ + { type: "vendor.future.event", detail: "ignored" }, + { + type: "content_block_start", + index: 0, + content_block: { type: "text", text: "partial" }, + }, + { type: "content_block_stop", index: 0 }, + ]), + { model: anthropicModel(), request: await prepared() }, + ), + ), + ); + assert.deepEqual( + truncated.map((event) => event.type), + ["text_delta", "error"], + ); + const truncatedTerminal = truncated.at(-1); + assert.equal(truncatedTerminal?.type, "error"); + if (truncatedTerminal?.type === "error") assert.equal(truncatedTerminal.partial, true); +}); diff --git a/packages/ai/test/auth.test.ts b/packages/ai/test/auth.test.ts index 3eb6f15..9f27dae 100644 --- a/packages/ai/test/auth.test.ts +++ b/packages/ai/test/auth.test.ts @@ -8,12 +8,15 @@ import { type ApiKeyAuthMethod, AuthError, type AuthContext, + type AuthEvent, + createProviderAuthentication, InMemoryCredentialStore, login, logout, type OAuthAuthMethod, type OAuthCredential, resolveProviderAuth, + safeProviderMessage, } from "../src/index.ts"; const providerId = "azure"; @@ -113,6 +116,27 @@ test("valid oauth resolves without refreshing and lists its tokens as secrets", assert.deepEqual(resolved.secretValues, ["current-access", "fresh-refresh"]); }); +test("authentication status checks do not refresh stored OAuth credentials", async () => { + const store = new InMemoryCredentialStore(); + const method = makeOAuthMethod(validOAuth()); + await login(store, providerId, expiringOAuth()); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { oauth: method }, + store, + context: makeContext(), + }); + + assert.deepEqual(await authentication.check(), { + phase: "authenticated", + method: "oauth", + source: "Azure OAuth", + }); + assert.equal(method.refreshCount, 0); + assert.equal((await store.read(providerId))?.type, "oauth"); +}); + test("expiring oauth refreshes exactly once across concurrent resolutions", async () => { const store = new InMemoryCredentialStore(); const method = makeOAuthMethod(validOAuth()); @@ -211,3 +235,347 @@ test("resolution failures surface invalid_auth with the provider named", async ( error.providerId === providerId, ); }); + +function source( + type: "environment" | "file" | "ambient" | "keyless", + available: boolean, + calls: string[], +) { + return { + type, + displayName: type, + resolve: async () => { + calls.push(type); + if (!available) return undefined; + const key = `${type}-secret`; + return { + auth: type === "keyless" ? {} : { apiKey: key }, + source: type, + secretValues: type === "keyless" ? [] : [key], + }; + }, + } as const; +} + +function makeInteraction(events: AuthEvent[], signal?: AbortSignal) { + return { + ...(signal === undefined ? {} : { signal }), + prompt: () => Promise.resolve("answer"), + notify: (event: AuthEvent) => events.push(event), + }; +} + +test("ambient authentication uses fixed environment, file, ambient, and keyless precedence", async () => { + const store = new InMemoryCredentialStore(); + const calls: string[] = []; + const methods = { + sources: [ + source("keyless", true, calls), + source("ambient", true, calls), + source("file", true, calls), + source("environment", false, calls), + ], + }; + + const resolved = await resolveProviderAuth(providerId, methods, store, makeContext()); + assert.equal(resolved.source, "file"); + assert.deepEqual(calls, ["environment", "file"]); +}); + +test("stored credential failure never falls back through ambient precedence", async () => { + const store = new InMemoryCredentialStore(); + const calls: string[] = []; + await login(store, providerId, { type: "api_key", key: "rejected-secret" }); + const rejecting: ApiKeyAuthMethod = { + displayName: "stored key", + resolve: () => Promise.resolve(undefined), + }; + + await assert.rejects( + resolveProviderAuth( + providerId, + { apiKey: rejecting, sources: [source("environment", true, calls)] }, + store, + makeContext(), + ), + (error) => error instanceof AuthError && error.code === "invalid_auth", + ); + assert.deepEqual(calls, []); +}); + +test("interactive authorization reports UI-neutral lifecycle states", async () => { + const events: AuthEvent[] = []; + const store = new InMemoryCredentialStore(); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Test OAuth", + login: async (interaction) => { + interaction.notify({ type: "auth_url", url: "https://login.example.test" }); + return validOAuth("interactive-access"); + }, + refresh: async (credential) => credential, + toAuth: (credential) => ({ apiKey: credential.access }), + }, + }, + store, + context: makeContext(), + }); + + const state = await authentication.login("oauth", makeInteraction(events)); + assert.equal(state.phase, "authenticated"); + assert.deepEqual( + events.map((event) => event.type), + ["state", "auth_url", "state"], + ); + assert.equal(JSON.stringify(events).includes("interactive-access"), false); +}); + +test("interactive authorization cancellation cannot persist late completion", async () => { + const store = new InMemoryCredentialStore(); + const controller = new AbortController(); + let complete!: (credential: OAuthCredential) => void; + const pending = new Promise((resolvePromise) => { + complete = resolvePromise; + }); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Test OAuth", + login: () => pending, + refresh: async (credential) => credential, + toAuth: () => ({}), + }, + }, + store, + context: makeContext(), + }); + + const operation = authentication.login("oauth", makeInteraction([], controller.signal)); + controller.abort(); + complete(validOAuth("late-access")); + await assert.rejects(operation, { name: "AbortError" }); + assert.equal(await store.read(providerId), undefined); + assert.equal(authentication.state().phase, "idle"); +}); + +test("newest concurrent authorization completion wins deterministically", async () => { + const store = new InMemoryCredentialStore(); + const completions: ((credential: OAuthCredential) => void)[] = []; + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Test OAuth", + login: () => + new Promise((resolvePromise) => { + completions.push(resolvePromise); + }), + refresh: async (credential) => credential, + toAuth: () => ({}), + }, + }, + store, + context: makeContext(), + }); + + const first = authentication.login("oauth", makeInteraction([])); + const second = authentication.login("oauth", makeInteraction([])); + completions[1]?.(validOAuth("newest-access")); + await second; + completions[0]?.(validOAuth("stale-access")); + await assert.rejects(first, /superseded/); + const stored = await store.read(providerId); + assert.equal(stored?.type === "oauth" && stored.access, "newest-access"); +}); + +test("logout wins a race with interactive authorization completion", async () => { + const store = new InMemoryCredentialStore(); + let complete!: (credential: OAuthCredential) => void; + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Test OAuth", + login: () => + new Promise((resolvePromise) => { + complete = resolvePromise; + }), + refresh: async (credential) => credential, + toAuth: () => ({}), + }, + }, + store, + context: makeContext(), + }); + + const authorization = authentication.login("oauth", makeInteraction([])); + await authentication.logout(); + complete(validOAuth("late-access")); + await assert.rejects(authorization, /superseded/); + assert.equal(await store.read(providerId), undefined); + assert.deepEqual(authentication.state(), { phase: "logged_out" }); +}); + +test("logout wins a race with an in-flight refresh", async () => { + const store = new InMemoryCredentialStore(); + await login(store, providerId, expiringOAuth()); + let complete!: (credential: OAuthCredential) => void; + let refreshStarted!: () => void; + const started = new Promise((resolvePromise) => { + refreshStarted = resolvePromise; + }); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Test OAuth", + refresh: () => { + refreshStarted(); + return new Promise((resolvePromise) => { + complete = resolvePromise; + }); + }, + toAuth: (credential) => ({ apiKey: credential.access }), + }, + }, + store, + context: makeContext(), + }); + + const resolution = authentication.resolve(); + await started; + const loggedOut = authentication.logout(); + complete(validOAuth("refreshed-after-logout")); + await assert.rejects(resolution, /Authentication changed/); + await loggedOut; + assert.equal(await store.read(providerId), undefined); + assert.deepEqual(authentication.state(), { phase: "logged_out" }); +}); + +test("failed refresh enters reauthentication state without ambient fallback", async () => { + const store = new InMemoryCredentialStore(); + await login(store, providerId, expiringOAuth()); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth", "environment"], + methods: { + oauth: { + displayName: "Test OAuth", + refresh: () => Promise.reject(new Error("old-refresh-token")), + toAuth: () => ({}), + }, + sources: [source("environment", true, [])], + }, + store, + context: makeContext(), + }); + + await assert.rejects(authentication.resolve(), (error) => { + assert.equal(String(error).includes("old-refresh-token"), false); + return error instanceof AuthError && error.code === "refresh_failed"; + }); + assert.deepEqual(authentication.state(), { + phase: "reauthentication_required", + method: "oauth", + }); +}); + +test("malformed provider auth and credential responses fail loudly", async () => { + const store = new InMemoryCredentialStore(); + const malformedResolution: ApiKeyAuthMethod = { + displayName: "Malformed", + resolve: () => + Promise.resolve({ + auth: { apiKey: "unredacted-secret" }, + source: "malformed", + secretValues: [], + }), + }; + await assert.rejects( + resolveProviderAuth(providerId, { apiKey: malformedResolution }, store, makeContext()), + (error) => error instanceof AuthError && error.code === "invalid_auth", + ); + + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Malformed OAuth", + login: () => Promise.resolve({ type: "oauth", access: "only" } as OAuthCredential), + refresh: async (credential) => credential, + toAuth: () => ({}), + }, + }, + store, + context: makeContext(), + }); + await assert.rejects(authentication.login("oauth", makeInteraction([]))); + assert.equal(await store.read(providerId), undefined); + + const malformedEventAuthentication = createProviderAuthentication({ + providerId, + declaredMethods: ["oauth"], + methods: { + oauth: { + displayName: "Malformed events", + login: async (interaction) => { + interaction.notify({ type: "auth_url" } as never); + return validOAuth(); + }, + refresh: async (credential) => credential, + toAuth: () => ({}), + }, + }, + store, + context: makeContext(), + }); + await assert.rejects( + malformedEventAuthentication.login("oauth", makeInteraction([])), + (error) => error instanceof AuthError && error.code === "invalid_auth", + ); +}); + +test("authentication states and diagnostics expose no credential values", async () => { + const store = new InMemoryCredentialStore(); + const secret = "diagnostic-secret"; + await login(store, providerId, { type: "api_key", key: secret }); + const authentication = createProviderAuthentication({ + providerId, + declaredMethods: ["file"], + methods: { apiKey: apiKeyMethod }, + store, + context: makeContext(), + }); + + const resolved = await authentication.resolve(); + assert.deepEqual(resolved.secretValues, [secret]); + assert.equal(JSON.stringify(authentication.state()).includes(secret), false); + assert.equal( + safeProviderMessage(`failed with ${secret}`, resolved.secretValues), + "failed with [REDACTED]", + ); + + const leakingSource: ApiKeyAuthMethod = { + displayName: "Leaking source", + resolve: () => + Promise.resolve({ auth: { apiKey: secret }, source: secret, secretValues: [secret] }), + }; + await assert.rejects( + resolveProviderAuth( + providerId, + { apiKey: leakingSource }, + new InMemoryCredentialStore(), + makeContext(), + ), + (error) => error instanceof AuthError && error.code === "invalid_auth", + ); +}); diff --git a/packages/ai/test/aws-auth.test.ts b/packages/ai/test/aws-auth.test.ts new file mode 100644 index 0000000..790dd7f --- /dev/null +++ b/packages/ai/test/aws-auth.test.ts @@ -0,0 +1,161 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + createAmazonBedrockProvider, + InMemoryCredentialStore, + login, + type AuthContext, + type ModelStreamEvent, +} from "../src/index.ts"; + +function context(values: Readonly>): AuthContext { + return { env: (name) => values[name], fileExists: () => Promise.resolve(false) }; +} + +async function collect(stream: AsyncIterable): Promise { + const events: ModelStreamEvent[] = []; + for await (const event of stream) events.push(event); + return events; +} + +function bedrockResponse(): Response { + return new Response(new Uint8Array(), { + status: 200, + headers: { "content-type": "application/vnd.amazon.eventstream" }, + }); +} + +test("Bedrock signs every dispatch through the AWS default credential chain", async () => { + let credentialCalls = 0; + const requests: { url: string; headers: Headers; body: string }[] = []; + const provider = createAmazonBedrockProvider({ + store: new InMemoryCredentialStore(), + context: context({ AWS_REGION: "us-east-1" }), + awsAuth: { + credentials: (options) => { + assert.deepEqual(options, {}); + return async () => { + credentialCalls += 1; + return { + accessKeyId: "AKIATESTACCESS", + secretAccessKey: "test-secret-access-key", + sessionToken: "test-session-token", + }; + }; + }, + }, + fetch: async (input, init) => { + requests.push({ + url: String(input), + headers: new Headers(init?.headers), + body: String(init?.body), + }); + return bedrockResponse(); + }, + }); + const model = (await provider.listModels())[0]; + assert.ok(model); + assert.equal(credentialCalls, 0); + + await collect(provider.stream({ modelId: model.modelId, messages: [] })); + assert.equal(credentialCalls, 1); + assert.equal(requests.length, 1); + assert.match(requests[0]?.headers.get("authorization") ?? "", /^AWS4-HMAC-SHA256 /); + assert.match(requests[0]?.headers.get("authorization") ?? "", /Credential=AKIATESTACCESS\//); + assert.equal(requests[0]?.headers.get("x-amz-security-token"), "test-session-token"); + assert.ok(requests[0]?.headers.get("x-amz-date")); + assert.equal(requests[0]?.url.includes("bedrock-runtime.us-east-1.amazonaws.com"), true); + assert.equal(requests[0]?.body.includes("test-secret-access-key"), false); +}); + +test("stored AWS profile is isolated and supplied to the default chain", async () => { + const store = new InMemoryCredentialStore(); + await login(store, "amazon-bedrock", { + type: "api_key", + env: { AWS_PROFILE: "engineering", AWS_REGION: "eu-west-1" }, + }); + let selectedProfile: string | undefined; + const provider = createAmazonBedrockProvider({ + store, + context: context({ AWS_PROFILE: "ignored", AWS_REGION: "us-east-1" }), + awsAuth: { + credentials: (options) => { + selectedProfile = options.profile; + return () => + Promise.resolve({ + accessKeyId: "AKIAPROFILE", + secretAccessKey: "profile-secret", + }); + }, + }, + }); + const resolved = await provider.authentication?.resolve(); + assert.equal(selectedProfile, "engineering"); + assert.equal(resolved?.env?.AWS_REGION, "eu-west-1"); + assert.equal(resolved?.source, "AWS profile engineering"); + assert.equal(typeof resolved?.auth.signRequest, "function"); +}); + +test("stored Bedrock bearer credentials never fall through to AWS signing", async () => { + const store = new InMemoryCredentialStore(); + await login(store, "amazon-bedrock", { + type: "api_key", + key: "bedrock-bearer", + env: { AWS_REGION: "us-west-2" }, + }); + let chainCalls = 0; + const provider = createAmazonBedrockProvider({ + store, + context: context({}), + awsAuth: { + credentials: () => { + chainCalls += 1; + return () => Promise.reject(new Error("must not resolve")); + }, + }, + }); + const resolved = await provider.authentication?.resolve(); + assert.equal(resolved?.auth.apiKey, "bedrock-bearer"); + assert.equal(resolved?.auth.signRequest, undefined); + assert.equal(chainCalls, 0); +}); + +test("Bedrock credential SDK promises are bounded by cancellation", async () => { + const provider = createAmazonBedrockProvider({ + store: new InMemoryCredentialStore(), + context: context({ AWS_REGION: "us-east-1" }), + awsAuth: { credentials: () => () => new Promise(() => undefined) }, + }); + assert.ok(provider.authentication); + const controller = new AbortController(); + const resolution = provider.authentication.resolve({ signal: controller.signal }); + controller.abort(); + await assert.rejects(resolution, { name: "AbortError" }); +}); + +test("Bedrock credential failures are explicit and do not produce unsigned requests", async () => { + let fetches = 0; + const provider = createAmazonBedrockProvider({ + store: new InMemoryCredentialStore(), + context: context({ AWS_REGION: "us-east-1" }), + awsAuth: { + credentials: () => () => Promise.reject(new Error("no AWS identity")), + }, + fetch: async () => { + fetches += 1; + return bedrockResponse(); + }, + }); + const model = (await provider.listModels())[0]; + assert.ok(model); + const events = await collect(provider.stream({ modelId: model.modelId, messages: [] })); + assert.equal(fetches, 0); + const terminal = events.at(-1); + assert.equal(terminal?.type, "error"); + assert.equal(terminal?.type === "error" && terminal.category, "authentication"); + assert.equal(JSON.stringify(events).includes("no AWS identity"), false); +}); diff --git a/packages/ai/test/aws-event-stream.test.ts b/packages/ai/test/aws-event-stream.test.ts new file mode 100644 index 0000000..784f841 --- /dev/null +++ b/packages/ai/test/aws-event-stream.test.ts @@ -0,0 +1,34 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { decodeAwsEventStream, MAX_AWS_EVENT_STREAM_FRAME_BYTES } from "../src/index.ts"; + +function stream(chunks: readonly Uint8Array[]): ReadableStream { + return new ReadableStream({ + start(controller) { + for (const chunk of chunks) controller.enqueue(chunk); + controller.close(); + }, + }); +} + +test("rejects oversized AWS frames from the prelude before buffering their body", async () => { + const prelude = new Uint8Array(16); + new DataView(prelude.buffer).setUint32(0, MAX_AWS_EVENT_STREAM_FRAME_BYTES + 1); + await assert.rejects( + Array.fromAsync(decodeAwsEventStream(stream([prelude]))), + /oversized lengths/, + ); +}); + +test("rejects oversized AWS preludes split across chunk boundaries", async () => { + const prelude = new Uint8Array(16); + new DataView(prelude.buffer).setUint32(0, MAX_AWS_EVENT_STREAM_FRAME_BYTES + 1); + await assert.rejects( + Array.fromAsync(decodeAwsEventStream(stream([prelude.subarray(0, 7), prelude.subarray(7)]))), + /oversized lengths/, + ); +}); diff --git a/packages/ai/test/azure-openai.test.ts b/packages/ai/test/azure-openai.test.ts index 5c2d185..b5b9f11 100644 --- a/packages/ai/test/azure-openai.test.ts +++ b/packages/ai/test/azure-openai.test.ts @@ -5,18 +5,24 @@ import assert from "node:assert/strict"; import test from "node:test"; import { - AuthError, type AuthContext, + AuthError, + AZURE_OPENAI_MODELS, azureOpenAiAuthMethod, + clampThinkingLevel, collectModelStream, createAzureOpenAiProvider, + encodeAzureOpenAiResponsesRequest, FakeModelProvider, + getStaticModelCatalog, InMemoryCredentialStore, login, - makeFakeModelInfo, type ModelStreamEvent, + makeFakeModelInfo, normalizeAzureBaseUrl, parseDeploymentMap, + prepareModelRequest, + THINKING_LEVELS, } from "../src/index.ts"; const usage = { inputTokens: 20, outputTokens: 30, cacheReadTokens: 100, cacheWriteTokens: 0 }; @@ -101,6 +107,10 @@ test("normalizes Azure base URLs and passes gateways through", () => { normalizeAzureBaseUrl("https://gateway.example.com/azure/"), "https://gateway.example.com/azure", ); + assert.equal( + normalizeAzureBaseUrl(`https://gateway.example.com/azure${"/".repeat(10_000)}`), + "https://gateway.example.com/azure", + ); assert.throws(() => normalizeAzureBaseUrl("not a url"), AuthError); }); @@ -140,6 +150,85 @@ test("parses the model-to-deployment map format", () => { assert.deepEqual(parseDeploymentMap("malformed,also=ok"), { also: "ok" }); }); +test("composes a prepared Azure request with deployment, headers, and API version", async () => { + const model = AZURE_OPENAI_MODELS.find((candidate) => candidate.modelId === "gpt-5"); + assert.ok(model); + const request = await prepareModelRequest(model, { + modelId: "gpt-5", + system: "Be concise.", + messages: [{ role: "user", content: [{ type: "text", text: "inspect" }] }], + tools: [ + { + name: "read_file", + description: "Read a file", + inputSchema: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + }, + }, + ], + toolChoice: "required", + thinkingLevel: "xhigh", + maxOutputTokens: 8, + }); + const encoded = encodeAzureOpenAiResponsesRequest(model, request, { + auth: { apiKey: "fixture-key", headers: { "x-azure-client": "fixture" } }, + source: "fixture", + env: { + AZURE_OPENAI_BASE_URL: "https://fixture.services.ai.azure.com/openai/v1/responses", + AZURE_OPENAI_API_VERSION: "2026-01-01-preview", + AZURE_OPENAI_DEPLOYMENT_NAME_MAP: "gpt-5=production-gpt-5", + }, + secretValues: ["fixture-key"], + }); + + assert.equal( + encoded.url, + "https://fixture.services.ai.azure.com/openai/v1/responses?api-version=2026-01-01-preview", + ); + assert.deepEqual(encoded.headers, { + "api-key": "fixture-key", + "x-azure-client": "fixture", + }); + assert.deepEqual(encoded.body, { + model: "production-gpt-5", + input: [{ role: "user", content: [{ type: "input_text", text: "inspect" }] }], + stream: true, + store: false, + instructions: "Be concise.", + max_output_tokens: 16, + tools: [ + { + type: "function", + name: "read_file", + description: "Read a file", + parameters: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + }, + strict: false, + }, + ], + tool_choice: "required", + reasoning: { effort: "high", summary: "auto" }, + include: ["reasoning.encrypted_content"], + }); +}); + +test("preserves proxy query settings while adding the Azure API version", async () => { + const { provider, requests } = await makeProvider(transcript, { + AZURE_OPENAI_BASE_URL: "https://gateway.example.com/azure?route=primary", + AZURE_OPENAI_API_VERSION: "2025-04-01-preview", + }); + await collectModelStream(provider.stream({ modelId: "gpt-5", messages: [] })); + assert.equal( + requests[0]?.url, + "https://gateway.example.com/azure/responses?route=primary&api-version=2025-04-01-preview", + ); +}); + test("streams from Azure with api-key header, versioned URL, and mapped deployment", async () => { const { provider, requests } = await makeProvider(transcript, { AZURE_OPENAI_RESOURCE_NAME: "myres", @@ -158,9 +247,9 @@ test("streams from Azure with api-key header, versioned URL, and mapped deployme assert.equal(request?.url, "https://myres.openai.azure.com/openai/v1/responses?api-version=v1"); assert.equal(request?.headers["api-key"], "azure-secret-key"); assert.equal(request?.body.model, "gpt-5.6-sol"); - assert.deepEqual(request?.body.reasoning, { effort: "xhigh" }); + assert.deepEqual(request?.body.reasoning, { effort: "high", summary: "auto" }); - assert.equal(events.length, 4); + assert.equal(events.length, 6); assert.equal(terminal.type, "completed"); if (terminal.type === "completed") { assert.equal(terminal.stopReason, "tool_use"); @@ -168,12 +257,73 @@ test("streams from Azure with api-key header, versioned URL, and mapped deployme } }); +test("retains Azure-specific replay provenance from the shared Responses stream", async () => { + const { provider } = await makeProvider( + [ + { type: "response.created", response: { id: "resp-azure" } }, + { + type: "response.output_item.added", + output_index: 0, + item: { type: "reasoning", id: "rs-azure" }, + }, + { type: "response.reasoning_summary_text.delta", output_index: 0, delta: "think" }, + { + type: "response.output_item.done", + output_index: 0, + item: { + type: "reasoning", + id: "rs-azure", + encrypted_content: "opaque-azure", + summary: [], + }, + }, + { + type: "response.completed", + response: { id: "resp-azure", status: "completed", usage: {} }, + }, + ], + { AZURE_OPENAI_RESOURCE_NAME: "myres" }, + ); + const { events } = await collectModelStream(provider.stream({ modelId: "gpt-5", messages: [] })); + assert.deepEqual(events[1], { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "azure-openai", + apiDialect: "azure-openai-responses", + modelId: "gpt-5", + signature: + '{"type":"reasoning","id":"rs-azure","encrypted_content":"opaque-azure","summary":[]}', + responseId: "resp-azure", + itemId: "rs-azure", + }); + assert.equal(events.at(-1)?.type, "completed"); +}); + test("exit gate: Azure and the fake provider produce identical canonical stream shapes", async () => { const canonical: readonly ModelStreamEvent[] = [ { type: "thinking_delta", text: "hmm" }, { type: "text_delta", text: "Hello" }, - { type: "tool_call", callId: "call-1", name: "shell", input: { command: "ls" } }, - { type: "completed", stopReason: "tool_use", usage: { ...usage, reasoningTokens: 0 } }, + { type: "tool_call_start", contentIndex: 0, callId: "call-1", name: "shell" }, + { + type: "tool_call_delta", + contentIndex: 0, + callId: "call-1", + argumentsDelta: '{"command":"ls"}', + }, + { + type: "tool_call", + contentIndex: 0, + callId: "call-1", + name: "shell", + input: { command: "ls" }, + }, + { + type: "completed", + stopReason: "tool_use", + usage: { ...usage, reasoningTokens: 0 }, + response: { providerId: "azure-openai", requestedModelId: "gpt-5" }, + }, ]; const fake = new FakeModelProvider({ models: [makeFakeModelInfo({ modelId: "gpt-5" })], @@ -210,7 +360,9 @@ test("HTTP failures terminate through the stream contract without leaking the ke const store = new InMemoryCredentialStore(); await login(store, "azure-openai", { type: "api_key", key: "azure-secret-key" }); const failingFetch = (async () => - new Response('{"error":{"message":"deployment not found"}}', { status: 404 })) as typeof fetch; + new Response('{"error":{"message":"azure-secret-key deployment not found"}}', { + status: 404, + })) as typeof fetch; const provider = createAzureOpenAiProvider({ store, context: makeContext({ AZURE_OPENAI_RESOURCE_NAME: "myres" }), @@ -224,6 +376,7 @@ test("HTTP failures terminate through the stream contract without leaking the ke if (terminal.type === "error") { assert.equal(terminal.code, "http_404"); assert.equal(terminal.retryable, false); + assert.equal(terminal.message, "Provider azure-openai returned 404"); assert.equal(terminal.message.includes("azure-secret-key"), false); } }); @@ -256,6 +409,7 @@ test("missing configuration surfaces a typed auth state through the stream", asy context: makeContext({}), fetch: (async () => new Response("")) as typeof fetch, }); + assert.deepEqual(provider.authentication?.methods, ["environment", "file"]); const { terminal } = await collectModelStream( provider.stream({ modelId: "gpt-5", messages: [] }), ); @@ -315,7 +469,10 @@ test("publishes the complete built-in Azure OpenAI model catalog", async () => { ); assert.equal(new Set(AZURE_OPENAI_MODELS.map((model) => model.modelId)).size, 38); assert.equal( - AZURE_OPENAI_MODELS.every((model) => model.providerId === "azure-openai"), + AZURE_OPENAI_MODELS.every( + (model) => + model.providerId === "azure-openai" && model.apiDialect === "azure-openai-responses", + ), true, ); assert.equal( @@ -327,3 +484,34 @@ test("publishes the complete built-in Azure OpenAI model catalog", async () => { { off: null, xhigh: "xhigh", max: "max" }, ); }); + +test("every generated Azure model encodes its declared reasoning map", async () => { + const models = getStaticModelCatalog("azure-openai-responses"); + for (const model of models) { + for (const requested of THINKING_LEVELS) { + const prepared = await prepareModelRequest(model, { + modelId: model.modelId, + messages: [], + thinkingLevel: requested, + }); + const encoded = encodeAzureOpenAiResponsesRequest(model, prepared, { + auth: { apiKey: "obviously-fake-key" }, + env: { AZURE_OPENAI_BASE_URL: "https://example.openai.azure.com/openai/v1" }, + source: "test", + secretValues: ["obviously-fake-key"], + }); + const clamp = clampThinkingLevel(model, requested); + assert.equal(prepared.thinkingLevel, clamp.effective); + if (clamp.effective === "off") { + assert.equal(encoded.body.reasoning, undefined, `${model.modelId}/${requested}`); + } else { + assert.deepEqual( + encoded.body.reasoning, + { effort: model.thinkingLevelMap?.[clamp.effective] ?? clamp.effective, summary: "auto" }, + `${model.modelId}/${requested}`, + ); + } + if (!model.reasoning) assert.equal(clamp.effective, "off"); + } + } +}); diff --git a/packages/ai/test/bedrock-converse-stream.test.ts b/packages/ai/test/bedrock-converse-stream.test.ts new file mode 100644 index 0000000..20940bc --- /dev/null +++ b/packages/ai/test/bedrock-converse-stream.test.ts @@ -0,0 +1,541 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + BedrockConverseStreamCodecError, + decodeBedrockConverseStream, + encodeBedrockConverseStreamRequest, + getStaticModelCatalog, + type ModelInfo, + type ModelRequest, + normalizeModelStream, + prepareModelRequest, +} from "../src/index.ts"; + +function bedrockModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "amazon-bedrock", + modelId: "us.anthropic.claude-sonnet-4-5-20250929-v1:0", + displayName: "Claude Sonnet 4.5", + apiDialect: "bedrock-converse-stream", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { + minimal: "1024", + low: "2048", + medium: "8192", + high: "16384", + xhigh: "16384", + max: "16384", + }, + contextWindow: 200_000, + maxOutputTokens: 64_000, + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.1, + cacheWriteUsdPerMTok: 1.25, + }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short", "long"], + }, + sampling: { supported: ["temperature", "topP"], customFields: ["stopSequences"] }, + compatibility: { + dialect: "bedrock-converse-stream", + supportsStrictTools: true, + supportsPromptCacheMarkers: true, + supportsThinkingSignatures: true, + }, + ...overrides, + }; +} + +const baseRequest: ModelRequest = { + modelId: "us.anthropic.claude-sonnet-4-5-20250929-v1:0", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], +}; + +async function prepared(request: ModelRequest = baseRequest, model = bedrockModel()) { + return prepareModelRequest(model, request); +} + +async function* events(items: readonly unknown[]) { + for (const item of items) yield item as Readonly>; +} + +test("generated Bedrock catalog declares codec capabilities explicitly", () => { + const models = getStaticModelCatalog("amazon-bedrock"); + const claude = models.find( + (model) => model.modelId === "us.anthropic.claude-sonnet-4-5-20250929-v1:0", + ); + const adaptive = models.find((model) => model.modelId === "global.anthropic.claude-sonnet-4-6"); + const nova = models.find((model) => model.modelId === "amazon.nova-lite-v1:0"); + assert.deepEqual(claude?.compatibility, { + dialect: "bedrock-converse-stream", + supportsStrictTools: true, + supportsPromptCacheMarkers: true, + supportsThinkingSignatures: true, + }); + assert.equal( + adaptive?.compatibility?.dialect === "bedrock-converse-stream" && + adaptive.compatibility.forceAdaptiveThinking, + true, + ); + assert.deepEqual(nova?.compatibility, { dialect: "bedrock-converse-stream" }); +}); + +test("encodes prepared history, images, tools, caching, reasoning, and signing inputs", async () => { + const model = bedrockModel(); + const image = new Uint8Array([1, 2, 3, 4]); + const sha256 = createHash("sha256").update(image).digest("hex"); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const request = await prepared( + { + modelId: model.modelId, + system: "Use evidence.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: image.length } }, + ], + }, + { + role: "assistant", + content: [ + { type: "thinking", text: "checked", signature: { ...identity, value: "sig-1" } }, + { + type: "thinking", + text: "", + signature: { ...identity, value: "AQID" }, + redacted: true, + }, + { type: "text", text: "calling" }, + ], + toolCalls: [{ callId: "call-1", name: "lookup", input: { key: "a", "": "removed" } }], + }, + { + role: "tool", + callId: "call-1", + name: "lookup", + content: [{ type: "text", text: "value" }], + isError: false, + }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { key: { type: "string" } }, + required: ["key"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "medium", + maxOutputTokens: 100, + toolChoice: "required", + sampling: { temperature: 0.2, topP: 0.8, custom: { stopSequences: ["END"] } }, + cache: { retention: "long" }, + metadata: { team: "search" }, + readBlob: async () => image, + }, + model, + ); + assert.equal(request.maxOutputTokens, 8_292); + const encoded = encodeBedrockConverseStreamRequest(model, request, { + region: "us-west-2", + authentication: { type: "sigv4" }, + }); + assert.equal( + encoded.url, + "https://bedrock-runtime.us-west-2.amazonaws.com/model/us.anthropic.claude-sonnet-4-5-20250929-v1%3A0/converse-stream", + ); + assert.deepEqual(encoded.signing, { service: "bedrock", region: "us-west-2" }); + assert.deepEqual(encoded.headers, { + accept: "application/vnd.amazon.eventstream", + "content-type": "application/json", + }); + assert.deepEqual(encoded.body, { + messages: [ + { + role: "user", + content: [{ text: "inspect" }, { image: { format: "png", source: { bytes: "AQIDBA==" } } }], + }, + { + role: "assistant", + content: [ + { reasoningContent: { reasoningText: { text: "checked", signature: "sig-1" } } }, + { reasoningContent: { redactedContent: "AQID" } }, + { text: "calling" }, + { toolUse: { toolUseId: "call-1", name: "lookup", input: { key: "a" } } }, + ], + }, + { + role: "user", + content: [ + { + toolResult: { + toolUseId: "call-1", + content: [{ text: "value" }], + status: "success", + }, + }, + { cachePoint: { type: "default", ttl: "1h" } }, + ], + }, + ], + system: [{ text: "Use evidence." }, { cachePoint: { type: "default", ttl: "1h" } }], + inferenceConfig: { maxTokens: 8_292, temperature: 0.2, topP: 0.8 }, + toolConfig: { + tools: [ + { + toolSpec: { + name: "lookup", + description: "Look up a value", + inputSchema: { + json: { + type: "object", + properties: { key: { type: "string" } }, + required: ["key"], + additionalProperties: false, + }, + }, + strict: true, + }, + }, + ], + toolChoice: { any: {} }, + }, + additionalModelRequestFields: { + stopSequences: ["END"], + thinking: { type: "enabled", budget_tokens: 7_268, display: "summarized" }, + anthropic_beta: ["interleaved-thinking-2025-05-14"], + }, + requestMetadata: { team: "search" }, + }); +}); + +test("uses ARN region routing and bearer authentication without signing", async () => { + const model = bedrockModel({ + modelId: + "arn:aws-us-gov:bedrock:us-gov-west-1:123456789012:application-inference-profile/example", + }); + const request = await prepared({ ...baseRequest, modelId: model.modelId }, model); + const encoded = encodeBedrockConverseStreamRequest(model, request, { + region: "us-east-1", + baseUrl: "https://private.example.test/runtime?route=one", + authentication: { type: "bearer", token: "token-value" }, + }); + assert.equal(encoded.signing, undefined); + assert.equal(encoded.headers.authorization, "Bearer token-value"); + assert.equal( + encoded.url, + "https://private.example.test/runtime/model/arn%3Aaws-us-gov%3Abedrock%3Aus-gov-west-1%3A123456789012%3Aapplication-inference-profile%2Fexample/converse-stream?route=one", + ); +}); + +test("encodes adaptive Claude reasoning without a token budget", async () => { + const model = bedrockModel({ + modelId: "global.anthropic.claude-sonnet-4-6", + thinkingLevelMap: { high: "high", xhigh: null, max: "max" }, + compatibility: { + dialect: "bedrock-converse-stream", + supportsStrictTools: true, + supportsPromptCacheMarkers: true, + supportsThinkingSignatures: true, + forceAdaptiveThinking: true, + }, + }); + const request = await prepared( + { modelId: model.modelId, messages: baseRequest.messages, thinkingLevel: "high" }, + model, + ); + const encoded = encodeBedrockConverseStreamRequest(model, request, { + region: "eu-central-1", + authentication: { type: "sigv4" }, + }); + assert.deepEqual(encoded.body.additionalModelRequestFields, { + thinking: { type: "adaptive", display: "summarized" }, + output_config: { effort: "high" }, + }); +}); + +test("decodes interleaved thinking, redaction, text, tools, usage, and routing", async () => { + const request = await prepared(); + const decoded = await Array.fromAsync( + decodeBedrockConverseStream( + events([ + { messageStart: { role: "assistant" } }, + { + contentBlockDelta: { + contentBlockIndex: 0, + delta: { reasoningContent: { text: "think", signature: "sig-" } }, + }, + }, + { + contentBlockDelta: { + contentBlockIndex: 0, + delta: { reasoningContent: { signature: "one" } }, + }, + }, + { contentBlockStop: { contentBlockIndex: 0 } }, + { + contentBlockDelta: { + contentBlockIndex: 1, + delta: { reasoningContent: { redactedContent: new Uint8Array([1, 2, 3]) } }, + }, + }, + { contentBlockStop: { contentBlockIndex: 1 } }, + { contentBlockDelta: { contentBlockIndex: 2, delta: { text: "done" } } }, + { contentBlockStop: { contentBlockIndex: 2 } }, + { + contentBlockStart: { + contentBlockIndex: 3, + start: { toolUse: { toolUseId: "call-1", name: "lookup" } }, + }, + }, + { + contentBlockDelta: { + contentBlockIndex: 3, + delta: { toolUse: { input: '{"key":"a"}' } }, + }, + }, + { contentBlockStop: { contentBlockIndex: 3 } }, + { messageStop: { stopReason: "tool_use" } }, + { + metadata: { + usage: { + inputTokens: 10, + outputTokens: 8, + cacheReadInputTokens: 4, + cacheWriteInputTokens: 2, + totalTokens: 18, + }, + metrics: { latencyMs: 42 }, + }, + }, + ]), + { + model: bedrockModel(), + request, + responseId: "request-1", + routedModelId: "profile/model", + }, + ), + ); + assert.deepEqual( + decoded.map((event) => event.type), + [ + "thinking_delta", + "replay_metadata", + "thinking_delta", + "replay_metadata", + "text_delta", + "tool_call_start", + "tool_call_delta", + "tool_call", + "completed", + ], + ); + assert.deepEqual(decoded[1], { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "amazon-bedrock", + apiDialect: "bedrock-converse-stream", + modelId: baseRequest.modelId, + signature: "sig-one", + }); + assert.deepEqual(decoded[3], { + type: "replay_metadata", + target: "thinking", + contentIndex: 1, + providerId: "amazon-bedrock", + apiDialect: "bedrock-converse-stream", + modelId: baseRequest.modelId, + signature: "AQID", + redacted: true, + }); + assert.deepEqual(decoded.at(-1), { + type: "completed", + stopReason: "tool_use", + usage: { + inputTokens: 10, + outputTokens: 8, + cacheReadTokens: 4, + cacheWriteTokens: 2, + reasoningTokens: 0, + costUsd: 0.000028899999999999998, + }, + response: { + providerId: "amazon-bedrock", + requestedModelId: baseRequest.modelId, + routedModelId: "profile/model", + responseId: "request-1", + nativeStopReason: "tool_use", + latencyMs: 42, + }, + }); +}); + +test("maps limits, policy stops, provider failures, and cancellation safely", async () => { + const request = await prepared(); + const limited = await Array.fromAsync( + decodeBedrockConverseStream( + events([ + { contentBlockDelta: { contentBlockIndex: 0, delta: { text: "partial" } } }, + { contentBlockStop: { contentBlockIndex: 0 } }, + { messageStop: { stopReason: "max_tokens" } }, + ]), + { model: bedrockModel(), request }, + ), + ); + const limitedTerminal = limited.at(-1); + assert.equal(limitedTerminal?.type, "completed"); + if (limitedTerminal?.type === "completed") assert.equal(limitedTerminal.partial, true); + + const policy = await Array.fromAsync( + decodeBedrockConverseStream(events([{ messageStop: { stopReason: "guardrail_intervened" } }]), { + model: bedrockModel(), + request, + }), + ); + assert.equal(policy[0]?.type === "error" && policy[0].category, "content_policy"); + + const secret = "secret-value"; + const failed = await Array.fromAsync( + decodeBedrockConverseStream(events([{ throttlingException: { message: `${secret} busy` } }]), { + model: bedrockModel(), + request, + secretValues: [secret], + }), + ); + assert.deepEqual(failed[0], { + type: "error", + code: "throttlingException", + message: "[REDACTED] busy", + retryable: true, + category: "rate_limit", + requestPhase: "streaming", + response: { providerId: "amazon-bedrock", requestedModelId: baseRequest.modelId }, + }); + + const controller = new AbortController(); + const cancelled = await prepared({ ...baseRequest, signal: controller.signal }); + async function* cancelling() { + yield { contentBlockDelta: { contentBlockIndex: 0, delta: { text: "partial" } } }; + controller.abort(); + yield { messageStop: { stopReason: "end_turn" } }; + } + const aborted = await Array.fromAsync( + decodeBedrockConverseStream(cancelling(), { model: bedrockModel(), request: cancelled }), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); +}); + +test("finalizes reasoning when Bedrock omits a content block stop", async () => { + const request = await prepared(); + const decoded = await Array.fromAsync( + decodeBedrockConverseStream( + events([ + { + contentBlockDelta: { + contentBlockIndex: 0, + delta: { reasoningContent: { redactedContent: "AQID" } }, + }, + }, + { messageStop: { stopReason: "end_turn" } }, + ]), + { model: bedrockModel(), request }, + ), + ); + assert.equal(decoded[1]?.type, "replay_metadata"); + if (decoded[1]?.type === "replay_metadata") { + assert.equal(decoded[1].signature, "AQID"); + assert.equal(decoded[1].redacted, true); + } + assert.equal(decoded[2]?.type, "completed"); +}); + +test("fails malformed events and normalizes truncation exactly once", async () => { + const request = await prepared(); + await assert.rejects( + Array.fromAsync( + decodeBedrockConverseStream(events([{ contentBlockStop: { contentBlockIndex: 4 } }]), { + model: bedrockModel(), + request, + }), + ), + BedrockConverseStreamCodecError, + ); + const normalized = await Array.fromAsync( + normalizeModelStream( + decodeBedrockConverseStream( + events([{ contentBlockDelta: { contentBlockIndex: 0, delta: { text: "partial" } } }]), + { model: bedrockModel(), request }, + ), + ), + ); + assert.deepEqual(normalized.at(-1), { + type: "error", + code: "provider_stream_truncated", + message: "provider ended the stream without a terminal event", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + partial: true, + }); +}); + +test("rejects unprepared requests and unsafe endpoint, auth, and metadata values", async () => { + const model = bedrockModel(); + assert.throws( + () => + encodeBedrockConverseStreamRequest(model, baseRequest as never, { + region: "us-east-1", + authentication: { type: "sigv4" }, + }), + /requires a prepared model request/, + ); + const request = await prepared({ ...baseRequest, metadata: { count: 2 } }); + assert.throws( + () => + encodeBedrockConverseStreamRequest(model, request, { + region: "us-east-1", + authentication: { type: "sigv4" }, + }), + /must be a string/, + ); + const safe = await prepared(); + assert.throws( + () => + encodeBedrockConverseStreamRequest(model, safe, { + region: "not-a-region", + authentication: { type: "sigv4" }, + }), + /region is invalid/, + ); + assert.throws( + () => + encodeBedrockConverseStreamRequest(model, safe, { + region: "us-east-1", + baseUrl: "https://user:pass@example.test", + authentication: { type: "sigv4" }, + }), + /unsupported URL data/, + ); +}); diff --git a/packages/ai/test/builtin-providers.test.ts b/packages/ai/test/builtin-providers.test.ts new file mode 100644 index 0000000..09710ad --- /dev/null +++ b/packages/ai/test/builtin-providers.test.ts @@ -0,0 +1,167 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + BUILTIN_PROVIDER_IDS, + createBuiltinProviders, + getStaticModelCatalog, + InMemoryCredentialStore, + listBuiltinCatalogProviders, + ProviderRegistry, +} from "../src/index.ts"; + +function context(read?: () => void) { + return { + env: () => { + read?.(); + return undefined; + }, + fileExists: () => { + read?.(); + return Promise.resolve(false); + }, + }; +} + +test("registers exactly all 41 planned provider identities", async () => { + let credentialReads = 0; + let fetches = 0; + const providers = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: context(() => { + credentialReads += 1; + }), + fetch: async () => { + fetches += 1; + throw new Error("unexpected network request"); + }, + }); + const ids = providers.map((provider) => provider.id); + assert.equal(ids.length, 41); + assert.equal(new Set(ids).size, 41); + assert.deepEqual([...ids].sort(), [...BUILTIN_PROVIDER_IDS].sort()); + assert.deepEqual( + [...ids].sort(), + listBuiltinCatalogProviders() + .map((provider) => provider.id) + .sort(), + ); + + const registry = new ProviderRegistry(); + for (const provider of providers) registry.register(provider); + for (const provider of providers) { + const models = await provider.listModels(); + assert.equal( + models.every((model) => model.providerId === provider.id), + true, + provider.id, + ); + const catalog = listBuiltinCatalogProviders().find((entry) => entry.id === provider.id); + assert.ok(catalog, provider.id); + if (catalog.catalogKind === "static") { + assert.ok(models.length > 0, provider.id); + assert.equal(models.length, getStaticModelCatalog(provider.id).length, provider.id); + } else { + assert.deepEqual(models, [], provider.id); + } + assert.equal(registry.get(provider.id), provider); + } + assert.equal(credentialReads, 0); + assert.equal(fetches, 0); +}); + +test("preserves catalog selected dialects and exact endpoint policies", async () => { + const providers = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: context(), + }); + const byId = new Map(providers.map((provider) => [provider.id, provider])); + + assert.deepEqual( + new Set((await byId.get("openai")?.listModels())?.map((model) => model.apiDialect)), + new Set(["openai-chat", "openai-responses"]), + ); + assert.deepEqual( + new Set((await byId.get("opencode")?.listModels())?.map((model) => model.apiDialect)), + new Set(["openai-chat", "openai-responses", "anthropic-messages", "google-generative-ai"]), + ); + assert.deepEqual( + new Set((await byId.get("opencode-go")?.listModels())?.map((model) => model.apiDialect)), + new Set(["openai-chat", "openai-responses", "anthropic-messages"]), + ); + for (const provider of providers) { + for (const model of await provider.listModels()) { + assert.equal( + model.compatibility?.dialect, + model.apiDialect, + `${provider.id}/${model.modelId}`, + ); + assert.ok(model.endpoint, `${provider.id}/${model.modelId}`); + } + } +}); + +test("keeps dynamic refresh explicit and enables Codex only with OAuth", async () => { + let fetches = 0; + const providers = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: context(), + fetch: async () => { + fetches += 1; + throw new Error("network should require explicit refresh"); + }, + }); + for (const id of ["github-copilot", "openrouter", "cloudflare-ai-gateway", "radius"]) { + const provider = providers.find((candidate) => candidate.id === id); + assert.ok(provider?.refreshModelCatalog, id); + assert.deepEqual(await provider.listModels(), [], id); + } + assert.equal(fetches, 0); + + const codex = providers.find((candidate) => candidate.id === "openai-codex"); + assert.ok(codex); + const codexModels = await codex.listModels(); + assert.ok(codexModels.length > 0); + assert.equal( + codexModels.every((model) => model.availability?.status !== "unavailable"), + true, + ); + assert.deepEqual(codex.authMethods, ["oauth"]); + assert.ok(codex.authentication); + + const bedrock = providers.find((candidate) => candidate.id === "amazon-bedrock"); + assert.ok(bedrock); + assert.deepEqual(bedrock.authMethods, ["environment", "file", "ambient"]); + assert.equal( + (await bedrock.listModels()).every((model) => model.availability?.status !== "unavailable"), + true, + ); +}); + +test("isolates regional provider registrations and credential ownership", async () => { + const providers = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: context(), + }); + const regional = listBuiltinCatalogProviders().filter( + (provider) => provider.regionFamily !== undefined, + ); + const identities = new Set(); + const endpoints = new Set(); + for (const entry of regional) { + const provider = providers.find((candidate) => candidate.id === entry.id); + assert.ok(provider, entry.id); + assert.ok(provider.authentication, entry.id); + const key = `${entry.regionFamily}/${entry.region}`; + assert.equal(identities.has(key), false, key); + identities.add(key); + const model = (await provider.listModels())[0]; + assert.ok(model?.endpoint, entry.id); + const endpoint = JSON.stringify(model.endpoint); + assert.equal(endpoints.has(`${entry.regionFamily}/${endpoint}`), false, entry.id); + endpoints.add(`${entry.regionFamily}/${endpoint}`); + } +}); diff --git a/packages/ai/test/catalog-refresh.test.ts b/packages/ai/test/catalog-refresh.test.ts new file mode 100644 index 0000000..ffd82e3 --- /dev/null +++ b/packages/ai/test/catalog-refresh.test.ts @@ -0,0 +1,153 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + createBuiltinProviders, + InMemoryCatalogStore, + InMemoryCredentialStore, + ProviderRegistry, +} from "../src/index.ts"; + +function source(context = 128_000) { + return { + openai: { + models: { + "gpt-5-refresh-test": { + id: "gpt-5-refresh-test", + name: "Refreshed model", + tool_call: true, + structured_output: true, + reasoning: true, + reasoning_options: [{ type: "effort", values: ["low", "high"] }], + modalities: { input: ["text", "image"] }, + limit: { context, output: 16_000 }, + cost: { input: 1, output: 2 }, + // Remote endpoint and headers must never become credential-routing policy. + baseUrl: "https://attacker.example", + headers: { Authorization: "injected" }, + }, + }, + }, + }; +} + +test("explicit static refresh validates live facts, pins policy, and restores offline", async () => { + const store = new InMemoryCatalogStore(); + let requests = 0; + let body = source(); + const providers = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: { + env: (name) => (name === "OPENAI_API_KEY" ? "obviously-fake-key" : undefined), + fileExists: async () => false, + }, + fetch: async (url, init) => { + requests++; + assert.equal(String(url), "https://models.dev/api.json"); + assert.equal(new Headers(init?.headers).has("authorization"), false); + return Response.json(body); + }, + }); + const registry = new ProviderRegistry({ catalogStore: store }); + const provider = providers.find((p) => p.id === "openai"); + assert.ok(provider); + registry.register(provider); + const deepseek = providers.find((p) => p.id === "deepseek"); + assert.ok(deepseek); + registry.register(deepseek); + await registry.listModels(); + assert.equal(requests, 0); + const refreshed = await registry.refresh({ configuredOnly: true }); + assert.deepEqual(refreshed.refreshedProviderIds, ["openai"]); + assert.equal(requests, 1); + const model = await registry.getModel("openai", "gpt-5-refresh-test"); + assert.deepEqual(model.endpoint, { type: "fixed", baseUrl: "https://api.openai.com/v1" }); + assert.equal(model.apiDialect, "openai-responses"); + assert.equal(model.headers, undefined); + assert.equal(model.thinkingLevelMap?.medium, null); + const snapshot = registry.catalogSnapshot("openai"); + assert.ok(snapshot); + body = source(1); + assert.equal((await registry.refresh({ providerId: "openai" })).errors.size, 1); + assert.deepEqual(registry.catalogSnapshot("openai"), snapshot); + const restored = new ProviderRegistry({ catalogStore: store }); + restored.register(provider); + await restored.restoreCatalogs(); + assert.deepEqual(await restored.getModel("openai", model.modelId), model); + assert.equal(requests, 2); + const dispatch = provider.streamModel?.bind(provider); + assert.ok(dispatch); + assert.throws( + () => + dispatch( + { ...model, endpoint: { type: "fixed", baseUrl: "https://attacker.example" } }, + {} as never, + ), + /reviewed endpoint/, + ); + await registry.dispose(); + await restored.dispose(); +}); + +test("cancelling a static refresh does not publish a candidate", async () => { + const controller = new AbortController(); + const provider = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: { env: () => undefined, fileExists: async () => false }, + fetch: async () => { + controller.abort(); + return Response.json(source()); + }, + }).find((p) => p.id === "openai"); + assert.ok(provider); + const registry = new ProviderRegistry(); + registry.register(provider); + await assert.rejects( + registry.refresh({ providerId: "openai", signal: controller.signal }), + /abort/i, + ); + assert.equal(registry.catalogSnapshot("openai"), undefined); + await registry.dispose(); +}); + +test("a newly refreshed Chat model dispatches through the registry", async () => { + let dispatched = false; + const model = source().openai.models["gpt-5-refresh-test"]; + const provider = createBuiltinProviders({ + store: new InMemoryCredentialStore(), + context: { + env: (name) => (name === "XAI_API_KEY" ? "obviously-fake-xai-key" : undefined), + fileExists: async () => false, + }, + fetch: async (url, init) => { + if (String(url) === "https://models.dev/api.json") { + return Response.json({ + xai: { + models: { + "grok-refresh-test": { ...model, id: "grok-refresh-test", reasoning: false }, + }, + }, + }); + } + assert.equal(String(url), "https://api.x.ai/v1/chat/completions"); + assert.equal(JSON.parse(String(init?.body)).model, "grok-refresh-test"); + dispatched = true; + return new Response( + 'data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}\n\ndata: [DONE]\n\n', + ); + }, + }).find((p) => p.id === "xai"); + assert.ok(provider); + const registry = new ProviderRegistry(); + registry.register(provider); + assert.equal((await registry.refresh({ providerId: "xai" })).errors.size, 0); + const events = await Array.fromAsync( + registry.stream("xai", { modelId: "grok-refresh-test", messages: [] }), + ); + assert.equal(dispatched, true); + assert.equal(events.at(-1)?.type, "completed"); + await registry.dispose(); +}); diff --git a/packages/ai/test/catalog-store.test.ts b/packages/ai/test/catalog-store.test.ts new file mode 100644 index 0000000..ad2a4e2 --- /dev/null +++ b/packages/ai/test/catalog-store.test.ts @@ -0,0 +1,120 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; + +import { + type CatalogSnapshot, + CatalogStoreError, + FileCatalogStore, + InMemoryCatalogStore, + makeFakeModelInfo, +} from "../src/index.ts"; + +function snapshot(providerId: string, generation = 1): CatalogSnapshot { + return { + version: 1, + providerId, + generation, + checkedAt: 200, + updatedAt: 100, + sourceUpdatedAt: 50, + etag: 'W/"catalog-1"', + source: { id: `${providerId}-models`, kind: "provider_api", revision: "2026-09-01" }, + models: [makeFakeModelInfo({ providerId, modelId: `model-${generation}` })], + }; +} + +test("in-memory catalog storage validates and clones provider snapshots", async () => { + const store = new InMemoryCatalogStore(); + const original = snapshot("dynamic"); + await store.write("dynamic", original); + + const first = await store.read("dynamic"); + assert.deepEqual(first, original); + assert.notEqual(first, original); + assert.notEqual(first?.models, original.models); + + const unsafe = { + ...original, + credential: "must-not-persist", + } as unknown as CatalogSnapshot; + await assert.rejects( + store.write("dynamic", unsafe), + (error) => error instanceof CatalogStoreError && /unknown field/.test(error.message), + ); + assert.deepEqual(await store.read("dynamic"), original); +}); + +test("file catalog storage keeps provider snapshots isolated and atomic", async () => { + const directory = await mkdtemp(join(tmpdir(), "axl-catalog-store-")); + try { + const store = new FileCatalogStore(directory); + await store.write("first", snapshot("first")); + await store.write("second", snapshot("second", 2)); + + const reloaded = new FileCatalogStore(directory); + assert.deepEqual(await reloaded.read("first"), snapshot("first")); + assert.deepEqual(await reloaded.read("second"), snapshot("second", 2)); + + const invalid = { + ...snapshot("first", 3), + models: [{ ...snapshot("first", 3).models[0], authorization: "secret" }], + } as unknown as CatalogSnapshot; + await assert.rejects(reloaded.write("first", invalid), CatalogStoreError); + assert.deepEqual(await reloaded.read("first"), snapshot("first")); + assert.deepEqual(await reloaded.read("second"), snapshot("second", 2)); + + const mode = (await stat(join(directory, "first.json"))).mode & 0o777; + if (process.platform !== "win32") assert.equal(mode, 0o600); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + +test("corrupt provider files fail loudly without affecting healthy providers", async () => { + const directory = await mkdtemp(join(tmpdir(), "axl-catalog-corrupt-")); + try { + const store = new FileCatalogStore(directory); + await store.write("healthy", snapshot("healthy")); + await writeFile(join(directory, "broken.json"), "{not-json\n", { mode: 0o600 }); + + await assert.rejects( + store.read("broken"), + (error) => error instanceof CatalogStoreError && /not valid JSON/.test(error.message), + ); + assert.deepEqual(await store.read("healthy"), snapshot("healthy")); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); + +test("cancelled lock waits never publish a delayed catalog write", async () => { + const directory = await mkdtemp(join(tmpdir(), "axl-catalog-cancel-")); + try { + const store = new FileCatalogStore(directory); + await store.write("dynamic", snapshot("dynamic")); + const lockPath = join(directory, "dynamic.json.lock"); + await writeFile(lockPath, `${process.pid} ${Date.now()}\n`, { mode: 0o600 }); + const controller = new AbortController(); + const pending = store.write("dynamic", snapshot("dynamic", 2), { + signal: controller.signal, + }); + + setTimeout(() => controller.abort(), 40); + await assert.rejects(pending, { name: "AbortError" }); + await rm(lockPath, { force: true }); + await new Promise((resolvePromise) => setTimeout(resolvePromise, 75)); + + const persisted = JSON.parse( + await readFile(join(directory, "dynamic.json"), "utf8"), + ) as CatalogSnapshot; + assert.equal(persisted.generation, 1); + } finally { + await rm(directory, { recursive: true, force: true }); + } +}); diff --git a/packages/ai/test/catalog.test.ts b/packages/ai/test/catalog.test.ts new file mode 100644 index 0000000..cbcb92c --- /dev/null +++ b/packages/ai/test/catalog.test.ts @@ -0,0 +1,237 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import { readFileSync } from "node:fs"; +import test from "node:test"; + +import { generateCatalog } from "../scripts/generate-catalog.ts"; +import { + GENERATED_CATALOG_PROVENANCE, + getStaticModelCatalog, + listBuiltinCatalogProviders, + ModelCatalogValidationError, + type ModelInfo, + STATIC_MODEL_CATALOG, + validateModelCatalog, +} from "../src/index.ts"; + +const validModel: ModelInfo = { + providerId: "test-provider", + modelId: "model-1", + displayName: "Model 1", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: false }, + reasoning: true, + thinkingLevelMap: { off: "none", high: "high", xhigh: null }, + contextWindow: 128_000, + maxOutputTokens: 16_000, + cost: { inputUsdPerMTok: 1, outputUsdPerMTok: 2 }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], + }, + endpoint: { type: "fixed", baseUrl: "https://example.test/v1" }, + availability: { status: "available" }, + compatibility: { dialect: "openai-chat", supportsStrictTools: true }, +}; + +test("generated catalog covers every planned provider identity", () => { + const providers = listBuiltinCatalogProviders(); + assert.equal(providers.length, 41); + assert.equal(new Set(providers.map((provider) => provider.id)).size, providers.length); + + const staticProviders = providers.filter((provider) => provider.catalogKind === "static"); + assert.deepEqual( + Object.keys(STATIC_MODEL_CATALOG).sort(), + staticProviders.map((provider) => provider.id).sort(), + ); + for (const provider of staticProviders) { + const models = getStaticModelCatalog(provider.id); + assert.ok(models.length > 0, `${provider.id} has no generated models`); + assert.equal( + models.every((model) => model.providerId === provider.id), + true, + ); + assert.equal( + models.every((model) => model.endpoint !== undefined), + true, + ); + } + assert.deepEqual(getStaticModelCatalog("openrouter"), []); + assert.deepEqual(getStaticModelCatalog("radius"), []); + assert.doesNotThrow(() => validateModelCatalog(Object.values(STATIC_MODEL_CATALOG).flat())); + + const openAiDialects = new Set(getStaticModelCatalog("openai").map((model) => model.apiDialect)); + assert.equal(openAiDialects.has("openai-chat"), true); + assert.equal(openAiDialects.has("openai-responses"), true); + + const azureModels = getStaticModelCatalog("azure-openai-responses"); + assert.equal( + azureModels.every( + (model) => + model.endpoint?.type === "template" && + model.endpoint.template === "https://{resource}.openai.azure.com/openai/v1" && + model.endpoint.variables.length === 1 && + model.endpoint.variables[0]?.name === "resource", + ), + true, + ); +}); + +test("static catalog access performs no network or credential work", () => { + const originalFetch = globalThis.fetch; + let fetchCalls = 0; + globalThis.fetch = (() => { + fetchCalls += 1; + throw new Error("network access is forbidden during static catalog reads"); + }) as typeof fetch; + try { + assert.ok(getStaticModelCatalog("openai").length > 0); + assert.equal(fetchCalls, 0); + } finally { + globalThis.fetch = originalFetch; + } +}); + +test("generated catalog matches the pre-refactor semantic baseline", () => { + const baseline = JSON.parse( + readFileSync(new URL("../catalog/semantic-baseline.json", import.meta.url), "utf8"), + ) as { + readonly providerCount: number; + readonly staticProviderCount: number; + readonly modelCount: number; + readonly stableSerializationSha256: string; + }; + const stableSerialization = JSON.stringify({ + provenance: GENERATED_CATALOG_PROVENANCE, + providers: listBuiltinCatalogProviders(), + catalog: STATIC_MODEL_CATALOG, + }); + + assert.equal(listBuiltinCatalogProviders().length, baseline.providerCount); + assert.equal(Object.keys(STATIC_MODEL_CATALOG).length, baseline.staticProviderCount); + assert.equal(Object.values(STATIC_MODEL_CATALOG).flat().length, baseline.modelCount); + assert.equal( + createHash("sha256").update(stableSerialization).digest("hex"), + baseline.stableSerializationSha256, + ); +}); + +test("generated catalog retains independent source provenance", () => { + assert.equal(GENERATED_CATALOG_PROVENANCE.sources[0]?.name, "models.dev"); + assert.match(GENERATED_CATALOG_PROVENANCE.sources[0]?.sha256 ?? "", /^[a-f0-9]{64}$/); + assert.equal( + GENERATED_CATALOG_PROVENANCE.sources.some((source) => + source.location.startsWith("https://developer.ant-ling.com/"), + ), + true, + ); +}); + +test("regional catalog identities and endpoints remain separated", () => { + const providers = listBuiltinCatalogProviders().filter( + (provider) => provider.regionFamily !== undefined, + ); + const identities = new Set( + providers.map((provider) => `${provider.regionFamily}/${provider.region}`), + ); + assert.equal(identities.size, providers.length); + + for (const family of new Set(providers.map((provider) => provider.regionFamily))) { + const endpoints = providers + .filter((provider) => provider.regionFamily === family) + .map((provider) => JSON.stringify(getStaticModelCatalog(provider.id)[0]?.endpoint)); + assert.equal(new Set(endpoints).size, endpoints.length, `${family} reuses a regional endpoint`); + } +}); + +test("catalog validation rejects unsafe and inconsistent metadata", () => { + assert.throws( + () => validateModelCatalog([validModel, validModel]), + (error) => + error instanceof ModelCatalogValidationError && error.message.includes("is duplicated"), + ); + assert.throws( + () => + validateModelCatalog([ + { + ...validModel, + apiDialect: "unknown-dialect", + capabilities: { ...validModel.capabilities, toolUse: "yes" }, + } as unknown as ModelInfo, + ]), + (error) => { + assert.ok(error instanceof ModelCatalogValidationError); + assert.match(error.message, /API dialect/); + assert.match(error.message, /capabilities/); + return true; + }, + ); + assert.throws( + () => + validateModelCatalog([ + { + ...validModel, + compatibility: { dialect: "anthropic-messages" }, + cost: { inputUsdPerMTok: -1, outputUsdPerMTok: 2 }, + cache: { + supported: false, + defaultRetention: "short", + supportedRetentions: ["short"], + }, + endpoint: { + type: "template", + template: "https://{account}.example.test", + variables: [ + { name: "account", setting: "apiKey", required: true }, + { name: "unused", setting: "unused", required: true }, + ], + }, + headers: { Authorization: "secret" }, + }, + ]), + (error) => { + assert.ok(error instanceof ModelCatalogValidationError); + assert.match(error.message, /pricing/); + assert.match(error.message, /cache/); + assert.match(error.message, /unsafe setting/); + assert.match(error.message, /does not use variable/); + assert.match(error.message, /compatibility dialect/); + assert.match(error.message, /unsafe static header/); + return true; + }, + ); +}); + +test("catalog artifacts deterministically match provider shards and overlays", () => { + const generated = generateCatalog(); + assert.equal(generated.files.size, 37); + for (const [path, expected] of generated.files) { + assert.equal(readFileSync(path, "utf8"), expected, path); + } +}); + +test("reasoning maps require an object with valid values and a supported level", () => { + for (const thinkingLevelMap of [ + 42, + [], + false, + { low: 1 }, + { high: " " }, + { off: null, minimal: null, low: null, medium: null, high: null, xhigh: null, max: null }, + ]) { + assert.throws( + () => validateModelCatalog([{ ...validModel, thinkingLevelMap } as unknown as ModelInfo]), + ModelCatalogValidationError, + ); + } + assert.doesNotThrow(() => validateModelCatalog([{ ...validModel, thinkingLevelMap: {} }])); + assert.doesNotThrow(() => + validateModelCatalog([ + { ...validModel, thinkingLevelMap: { off: null, low: "low", high: "high", xhigh: null } }, + ]), + ); +}); diff --git a/packages/ai/test/cloud-auth.test.ts b/packages/ai/test/cloud-auth.test.ts new file mode 100644 index 0000000..11d51c1 --- /dev/null +++ b/packages/ai/test/cloud-auth.test.ts @@ -0,0 +1,215 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + createAzureOpenAiResponsesProvider, + createGoogleVertexProvider, + InMemoryCredentialStore, + login, + type AuthContext, + vertexRequestPolicy, +} from "../src/index.ts"; + +function context( + values: Readonly>, + files: readonly string[] = [], +): AuthContext { + return { + env: (name) => values[name], + fileExists: (path) => Promise.resolve(files.includes(path)), + }; +} + +test("Azure Entra acquires a scoped token lazily and asks again on later resolution", async () => { + let calls = 0; + const provider = createAzureOpenAiResponsesProvider({ + store: new InMemoryCredentialStore(), + context: context({ AZURE_OPENAI_RESOURCE_NAME: "sample" }), + cloudAuth: { + azureCredential: () => ({ + getToken: async (scope, options) => { + calls += 1; + assert.equal(scope, "https://cognitiveservices.azure.com/.default"); + assert.equal(options?.abortSignal?.aborted, false); + return { token: `entra-token-${calls}`, expiresOnTimestamp: Date.now() + 3_600_000 }; + }, + }), + }, + }); + + assert.equal(calls, 0); + const first = await provider.authentication?.resolve(); + const second = await provider.authentication?.resolve(); + assert.equal(first?.auth.headers?.authorization, "Bearer entra-token-1"); + assert.equal(second?.auth.headers?.authorization, "Bearer entra-token-2"); + assert.equal(first?.env?.AZURE_OPENAI_BASE_URL, "https://sample.openai.azure.com/openai/v1"); + assert.deepEqual(first?.secretValues, ["entra-token-1"]); +}); + +test("Azure exposes interactive API-key login with endpoint settings", async () => { + const store = new InMemoryCredentialStore(); + const provider = createAzureOpenAiResponsesProvider({ + store, + context: context({}), + }); + assert.ok(provider.authentication); + const answers = ["azure-login-key", "https://sample.openai.azure.com/openai/v1"]; + const state = await provider.authentication.login("api_key", { + signal: new AbortController().signal, + prompt: async () => answers.shift() ?? "", + notify: () => undefined, + }); + assert.equal(state.phase, "authenticated"); + const stored = await store.read("azure-openai-responses"); + assert.equal(stored?.type, "api_key"); + assert.equal( + stored?.type === "api_key" ? stored.env?.AZURE_OPENAI_BASE_URL : undefined, + "https://sample.openai.azure.com/openai/v1", + ); +}); + +test("stored Azure credentials never fall through to Entra", async () => { + const store = new InMemoryCredentialStore(); + await login(store, "azure-openai-responses", { type: "api_key", key: "bad-stored-key" }); + let entraCalls = 0; + const provider = createAzureOpenAiResponsesProvider({ + store, + context: context({ AZURE_OPENAI_RESOURCE_NAME: "sample" }), + cloudAuth: { + azureCredential: () => ({ + getToken: async () => { + entraCalls += 1; + return { token: "entra-token" }; + }, + }), + }, + }); + const resolved = await provider.authentication?.resolve(); + assert.equal(resolved?.auth.apiKey, "bad-stored-key"); + assert.equal(entraCalls, 0); +}); + +test("Vertex credential SDK promises are bounded by cancellation", async () => { + const provider = createGoogleVertexProvider({ + store: new InMemoryCredentialStore(), + context: context({ GOOGLE_CLOUD_LOCATION: "us-central1" }), + cloudAuth: { + googleAuth: () => ({ + getProjectId: () => new Promise(() => undefined), + getClient: () => new Promise(() => undefined), + }), + }, + }); + assert.ok(provider.authentication); + const controller = new AbortController(); + const resolution = provider.authentication.resolve({ signal: controller.signal }); + controller.abort(); + await assert.rejects(resolution, { name: "AbortError" }); +}); + +test("Vertex ADC acquires tokens and discovers a project without persisting them", async () => { + let tokenCalls = 0; + let optionsSeen: Readonly> | undefined; + const provider = createGoogleVertexProvider({ + store: new InMemoryCredentialStore(), + context: context({ GOOGLE_CLOUD_LOCATION: "us-central1" }), + cloudAuth: { + googleAuth: (options) => { + optionsSeen = options; + return { + getProjectId: () => Promise.resolve("discovered-project"), + getClient: () => + Promise.resolve({ + getAccessToken: () => { + tokenCalls += 1; + return Promise.resolve({ token: `google-token-${tokenCalls}` }); + }, + }), + }; + }, + }, + }); + + assert.equal(tokenCalls, 0); + const first = await provider.authentication?.resolve(); + const second = await provider.authentication?.resolve(); + assert.equal(first?.auth.headers?.authorization, "Bearer google-token-1"); + assert.equal(second?.auth.headers?.authorization, "Bearer google-token-2"); + assert.equal(first?.env?.GOOGLE_CLOUD_PROJECT, "discovered-project"); + assert.equal(first?.env?.GOOGLE_VERTEX_CREDENTIAL_TYPE, "adc"); + assert.deepEqual(optionsSeen, { + scopes: "https://www.googleapis.com/auth/cloud-platform", + }); + assert.ok(first); + assert.deepEqual(vertexRequestPolicy(first), { + credential: { type: "adc", accessToken: "google-token-1" }, + project: "discovered-project", + location: "us-central1", + }); +}); + +test("Vertex service account selection validates its file and remains provider scoped", async () => { + const store = new InMemoryCredentialStore(); + await login(store, "google-vertex", { + type: "api_key", + env: { + GOOGLE_VERTEX_CREDENTIAL_TYPE: "service_account", + GOOGLE_APPLICATION_CREDENTIALS: "/secure/service-account.json", + GOOGLE_CLOUD_PROJECT: "stored-project", + GOOGLE_CLOUD_LOCATION: "global", + }, + }); + let optionsSeen: Readonly> | undefined; + const provider = createGoogleVertexProvider({ + store, + context: context({}, ["/secure/service-account.json"]), + cloudAuth: { + googleAuth: (options) => { + optionsSeen = options; + return { + getProjectId: () => Promise.resolve("ignored-project"), + getClient: () => + Promise.resolve({ getAccessToken: () => Promise.resolve("service-token") }), + }; + }, + }, + }); + const resolved = await provider.authentication?.resolve(); + assert.deepEqual(optionsSeen, { + scopes: "https://www.googleapis.com/auth/cloud-platform", + keyFilename: "/secure/service-account.json", + projectId: "stored-project", + }); + assert.ok(resolved); + assert.deepEqual(vertexRequestPolicy(resolved), { + credential: { + type: "service_account", + accessToken: "service-token", + credentialsFile: "/secure/service-account.json", + }, + project: "stored-project", + location: "global", + }); + assert.equal(JSON.stringify(provider.authentication?.state()).includes("service-token"), false); +}); + +test("Vertex file source fails explicitly when its configured credential file is missing", async () => { + const provider = createGoogleVertexProvider({ + store: new InMemoryCredentialStore(), + context: context({ + GOOGLE_APPLICATION_CREDENTIALS: "/missing.json", + GOOGLE_CLOUD_PROJECT: "project", + GOOGLE_CLOUD_LOCATION: "us-central1", + }), + cloudAuth: { + googleAuth: () => { + throw new Error("must not instantiate GoogleAuth"); + }, + }, + }); + assert.ok(provider.authentication); + await assert.rejects(provider.authentication.resolve(), /does not identify a readable file/); +}); diff --git a/packages/ai/test/credentials.test.ts b/packages/ai/test/credentials.test.ts index ea0feea..2ad85ed 100644 --- a/packages/ai/test/credentials.test.ts +++ b/packages/ai/test/credentials.test.ts @@ -130,6 +130,22 @@ test("in-memory store matches the contract", async () => { assert.equal(await store.read("azure"), undefined); }); +test("every store rejects malformed credentials before writing", async (context) => { + const stores = [ + new InMemoryCredentialStore(), + new FileCredentialStore(await temporaryStorePath(context)), + ]; + for (const store of stores) { + await assert.rejects( + store.modify("azure", () => + Promise.resolve({ type: "oauth", access: "only" } as unknown as Credential), + ), + CredentialStoreError, + ); + assert.equal(await store.read("azure"), undefined); + } +}); + test("credentialSecretValues covers keys and oauth tokens", () => { assert.deepEqual(credentialSecretValues(apiKeyCredential), ["secret-key"]); assert.deepEqual(credentialSecretValues(oauthCredential), ["access-token", "refresh-token"]); diff --git a/packages/ai/test/deepseek-provider.test.ts b/packages/ai/test/deepseek-provider.test.ts new file mode 100644 index 0000000..542e68a --- /dev/null +++ b/packages/ai/test/deepseek-provider.test.ts @@ -0,0 +1,292 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + AuthError, + collectModelStream, + createDeepSeekProvider, + DEEPSEEK_BASE_URL, + DEEPSEEK_MODELS, + InMemoryCredentialStore, + type ModelStreamEvent, + ProviderRegistry, +} from "../src/index.ts"; + +const model = DEEPSEEK_MODELS[0]; +if (model === undefined) throw new Error("DeepSeek catalog is empty"); + +function context(environment: Readonly> = {}) { + return { + env: (name: string) => environment[name], + fileExists: () => Promise.resolve(false), + }; +} + +function streamResponse(values: readonly unknown[]): Response { + const body = values + .map((value) => `data: ${typeof value === "string" ? value : JSON.stringify(value)}\n\n`) + .join(""); + return new Response(body, { + status: 200, + headers: { "content-type": "text/event-stream" }, + }); +} + +async function events( + stream: AsyncIterable, +): Promise { + const result: ModelStreamEvent[] = []; + for await (const event of stream) result.push(event); + return result; +} + +test("constructs and lists DeepSeek without credential or network work", async () => { + let environmentReads = 0; + let fetches = 0; + const provider = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: { + env: () => { + environmentReads += 1; + return undefined; + }, + fileExists: () => Promise.resolve(false), + }, + fetch: async () => { + fetches += 1; + throw new Error("unexpected network request"); + }, + }); + + assert.equal(provider.id, "deepseek"); + assert.equal(provider.displayName, "DeepSeek"); + assert.deepEqual(provider.authMethods, ["environment", "file"]); + assert.equal("refreshModels" in provider, false); + assert.deepEqual(await provider.listModels(), DEEPSEEK_MODELS); + assert.equal(environmentReads, 0); + assert.equal(fetches, 0); +}); + +test("resolves stored DeepSeek credentials before environment credentials", async () => { + const store = new InMemoryCredentialStore(); + await store.modify("deepseek", () => Promise.resolve({ type: "api_key", key: "stored-secret" })); + const provider = createDeepSeekProvider({ + store, + context: context({ DEEPSEEK_API_KEY: "environment-secret" }), + }); + + assert.deepEqual(await provider.authentication?.resolve(), { + auth: { apiKey: "stored-secret" }, + source: "stored credential", + secretValues: ["stored-secret"], + }); +}); + +test("does not fall through when a stored DeepSeek credential has no key", async () => { + const store = new InMemoryCredentialStore(); + await store.modify("deepseek", () => Promise.resolve({ type: "api_key" })); + const provider = createDeepSeekProvider({ + store, + context: context({ DEEPSEEK_API_KEY: "environment-secret" }), + }); + + const authentication = provider.authentication; + if (authentication === undefined) throw new Error("DeepSeek authentication is missing"); + await assert.rejects( + authentication.resolve(), + (error: unknown) => error instanceof AuthError && error.code === "invalid_auth", + ); +}); + +test("supports provider-owned DeepSeek API key login", async () => { + const store = new InMemoryCredentialStore(); + const provider = createDeepSeekProvider({ store, context: context() }); + const prompts: unknown[] = []; + + const state = await provider.authentication?.login("api_key", { + prompt: (prompt) => { + prompts.push(prompt); + return Promise.resolve("entered-secret"); + }, + notify: () => undefined, + }); + + assert.equal(state?.phase, "authenticated"); + assert.equal(prompts.length, 1); + assert.deepEqual(await store.read("deepseek"), { type: "api_key", key: "entered-secret" }); +}); + +test("registers DeepSeek and streams through the prepared Chat transport", async () => { + let requestUrl = ""; + let requestInit: RequestInit | undefined; + const fetchImpl: typeof fetch = async (input, init) => { + requestUrl = String(input); + requestInit = init; + return streamResponse([ + { id: "response-1", model: "deepseek-routed", choices: [{ delta: { content: "ok" } }] }, + { + usage: { prompt_tokens: 2, completion_tokens: 1 }, + choices: [{ delta: {}, finish_reason: "stop" }], + }, + "[DONE]", + ]); + }; + const provider = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "deepseek-secret" }), + fetch: fetchImpl, + now: () => 100, + }); + const registry = new ProviderRegistry(); + registry.register(provider); + + const result = await collectModelStream( + registry.stream("deepseek", { modelId: model.modelId, messages: [] }), + ); + + assert.equal(requestUrl, `${DEEPSEEK_BASE_URL}/chat/completions`); + assert.equal(new Headers(requestInit?.headers).get("authorization"), "Bearer deepseek-secret"); + assert.equal(new Headers(requestInit?.headers).get("accept"), "text/event-stream"); + assert.deepEqual(JSON.parse(String(requestInit?.body)), { + model: model.modelId, + messages: [], + stream: true, + stream_options: { include_usage: true }, + }); + assert.deepEqual(result.events[0], { type: "text_delta", text: "ok", contentIndex: 0 }); + assert.equal(result.terminal.type, "completed"); + assert.deepEqual(result.terminal.response, { + providerId: "deepseek", + requestedModelId: model.modelId, + routedModelId: "deepseek-routed", + responseId: "response-1", + nativeStopReason: "stop", + latencyMs: 0, + }); +}); + +test("retries bounded prestream failures and honors retry guidance", async () => { + let fetches = 0; + const fetchImpl: typeof fetch = async () => { + fetches += 1; + if (fetches === 1) { + return new Response("busy", { status: 429, headers: { "retry-after": "0" } }); + } + return streamResponse([{ choices: [{ delta: {}, finish_reason: "stop" }] }, "[DONE]"]); + }; + const provider = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret" }), + fetch: fetchImpl, + }); + + const completed = await events( + provider.stream({ modelId: model.modelId, messages: [], maxRetries: 1 }), + ); + assert.equal(fetches, 2); + assert.equal(completed.at(-1)?.type, "completed"); + + const limited = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret" }), + fetch: async () => new Response("busy", { status: 429, headers: { "retry-after": "3" } }), + }); + assert.deepEqual( + await events(limited.stream({ modelId: model.modelId, messages: [], maxRetries: 0 })), + [ + { + type: "error", + code: "http_429", + message: "Provider deepseek returned 429", + retryable: true, + category: "rate_limit", + requestPhase: "awaiting_response", + retryAfterMs: 3_000, + }, + ], + ); +}); + +test("reports cancellation, timeout, and redacted transport failures", async () => { + const cancelled = new AbortController(); + cancelled.abort(); + const provider = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret-value" }), + fetch: async () => { + throw new Error("should not fetch"); + }, + }); + assert.deepEqual( + await events( + provider.stream({ modelId: model.modelId, messages: [], signal: cancelled.signal }), + ), + [{ type: "aborted" }], + ); + + const timeoutProvider = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret-value" }), + fetch: (_input, init) => + new Promise((_resolve, reject) => { + const signal = init?.signal; + const guard = setTimeout(() => reject(new Error("timeout signal did not fire")), 100); + const abort = () => { + clearTimeout(guard); + reject(signal?.reason); + }; + if (signal?.aborted) abort(); + else signal?.addEventListener("abort", abort, { once: true }); + }), + }); + const timedOut = await events( + timeoutProvider.stream({ modelId: model.modelId, messages: [], timeoutMs: 1 }), + ); + assert.deepEqual(timedOut, [ + { + type: "error", + code: "provider_timeout", + message: "Provider deepseek request timed out", + retryable: true, + category: "timeout", + requestPhase: "before_dispatch", + }, + ]); + + let retryAttempts = 0; + const retryController = new AbortController(); + const retrying = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret-value" }), + fetch: async () => { + retryAttempts += 1; + retryController.abort(); + return new Response("busy", { status: 429, headers: { "retry-after": "30" } }); + }, + }); + assert.deepEqual( + await events( + retrying.stream({ modelId: model.modelId, messages: [], signal: retryController.signal }), + ), + [{ type: "aborted" }], + ); + assert.equal(retryAttempts, 1); + + const failed = createDeepSeekProvider({ + store: new InMemoryCredentialStore(), + context: context({ DEEPSEEK_API_KEY: "secret-value" }), + fetch: async () => { + throw new Error("credential secret-value rejected"); + }, + }); + const failedEvents = await events( + failed.stream({ modelId: model.modelId, messages: [], maxRetries: 0 }), + ); + assert.equal(failedEvents[0]?.type, "error"); + if (failedEvents[0]?.type !== "error") throw new Error("Expected transport error"); + assert.equal(failedEvents[0].message, "credential [REDACTED] rejected"); + assert.equal(failedEvents[0].category, "network"); +}); diff --git a/packages/ai/test/gateway-messages.test.ts b/packages/ai/test/gateway-messages.test.ts new file mode 100644 index 0000000..bf141da --- /dev/null +++ b/packages/ai/test/gateway-messages.test.ts @@ -0,0 +1,504 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + decodeGatewayMessagesStream, + encodeGatewayMessagesRequest, + GatewayMessagesCodecError, + normalizeModelStream, + prepareModelRequest, + type ModelInfo, + type ModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function gatewayModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "radius", + modelId: "auto", + displayName: "Radius Auto", + apiDialect: "gateway-messages", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { high: "high" }, + contextWindow: 128_000, + maxOutputTokens: 16_384, + cost: { inputUsdPerMTok: 0, outputUsdPerMTok: 0 }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], + }, + compatibility: { dialect: "gateway-messages", supportsStrictTools: true }, + ...overrides, + }; +} + +const baseRequest: ModelRequest = { + modelId: "auto", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], +}; + +async function prepared(request: ModelRequest = baseRequest, model = gatewayModel()) { + return prepareModelRequest(model, request); +} + +async function* frames(items: readonly (string | Record)[]) { + for (const item of items) { + yield { data: typeof item === "string" ? item : JSON.stringify(item) } satisfies SseFrame; + } +} + +const gatewayUsage = { + input: 10, + output: 5, + cacheRead: 3, + cacheWrite: 2, + reasoning: 1, + totalTokens: 20, + cost: { input: 0.01, output: 0.2, cacheRead: 0.003, cacheWrite: 0.004, total: 0.217 }, +}; + +test("encodes prepared Gateway context, images, replay, strict tools, and routing metadata", async () => { + const model = gatewayModel(); + const image = new Uint8Array([1, 2, 3]); + const sha256 = createHash("sha256").update(image).digest("hex"); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const request = await prepared({ + modelId: model.modelId, + system: "Route carefully.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: image.length } }, + ], + }, + { + role: "assistant", + origin: identity, + continuation: { ...identity, responseId: "response-old" }, + content: [ + { type: "thinking", text: "reason", signature: { ...identity, value: "think-sig" } }, + { type: "text", text: "calling", signature: { ...identity, value: "text-sig" } }, + ], + toolCalls: [ + { + callId: "call_1", + name: "lookup", + input: { query: "pi" }, + signature: { ...identity, value: "tool-sig" }, + continuation: { ...identity, namespace: "functions" }, + }, + ], + }, + { + role: "tool", + callId: "call_1", + name: "lookup", + content: [{ type: "text", text: "result" }], + isError: false, + }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "high", + maxOutputTokens: 100, + toolChoice: "required", + cache: { retention: "short", sessionId: "session-1" }, + metadata: { route: "quality", tenant: 7 }, + readBlob: async () => image, + }); + + assert.deepEqual(encodeGatewayMessagesRequest(model, request).body, { + model: "auto", + context: { + systemPrompt: "Route carefully.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "image", data: "AQID", mimeType: "image/png" }, + ], + timestamp: 0, + }, + { + role: "assistant", + content: [ + { type: "thinking", thinking: "reason", thinkingSignature: "think-sig" }, + { type: "text", text: "calling", textSignature: "text-sig" }, + { + type: "toolCall", + id: "call_1", + name: "lookup", + arguments: { query: "pi" }, + thoughtSignature: "tool-sig", + namespace: "functions", + }, + ], + api: "pi-messages", + provider: "radius", + model: "auto", + responseId: "response-old", + usage: { + input: 0, + output: 0, + cacheRead: 0, + cacheWrite: 0, + totalTokens: 0, + cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0, total: 0 }, + }, + stopReason: "toolUse", + timestamp: 0, + }, + { + role: "toolResult", + toolCallId: "call_1", + toolName: "lookup", + content: [{ type: "text", text: "result" }], + isError: false, + timestamp: 0, + }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + parameters: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + additionalProperties: false, + }, + constrainedSampling: { type: "json_schema", strict: "require" }, + }, + ], + }, + options: { + reasoning: "high", + maxTokens: 100, + toolChoice: "required", + cacheRetention: "short", + sessionId: "session-1", + metadata: { route: "quality", tenant: 7 }, + }, + }); +}); + +test("decodes text, reasoning, tools, signatures, usage, cost, and routed identity", async () => { + const request = await prepared({ + ...baseRequest, + tools: [{ name: "lookup", description: "Lookup", inputSchema: { type: "object" } }], + }); + const decoded = await Array.fromAsync( + decodeGatewayMessagesStream( + frames([ + { type: "start" }, + { type: "thinking_start", contentIndex: 0 }, + { type: "thinking_delta", contentIndex: 0, delta: "why" }, + { + type: "thinking_end", + contentIndex: 0, + content: "why", + contentSignature: "think-sig", + }, + { type: "text_start", contentIndex: 1 }, + { type: "text_delta", contentIndex: 1, delta: "answer" }, + { + type: "text_end", + contentIndex: 1, + content: "answer", + contentSignature: "text-sig", + }, + { type: "toolcall_start", contentIndex: 2, id: "call_2", toolName: "lookup" }, + { type: "toolcall_delta", contentIndex: 2, delta: '{"query":' }, + { type: "toolcall_delta", contentIndex: 2, delta: '"pi"}' }, + { + type: "toolcall_end", + contentIndex: 2, + toolCall: { + type: "toolCall", + id: "call_2", + name: "lookup", + arguments: { query: "pi" }, + thoughtSignature: "tool-sig", + namespace: "functions", + }, + }, + { + type: "done", + reason: "toolUse", + nativeStopReason: "upstream_tool_calls", + usage: gatewayUsage, + requestedModelId: "auto", + routedModelId: "anthropic/claude-sonnet", + responseId: "response-1", + }, + ]), + { model: gatewayModel(), request, startedAtMs: 10, now: () => 52 }, + ), + ); + + assert.deepEqual(decoded, [ + { type: "thinking_delta", text: "why", contentIndex: 0 }, + { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "radius", + apiDialect: "gateway-messages", + modelId: "auto", + signature: "think-sig", + }, + { type: "text_delta", text: "answer", contentIndex: 1 }, + { + type: "replay_metadata", + target: "text", + contentIndex: 1, + providerId: "radius", + apiDialect: "gateway-messages", + modelId: "auto", + signature: "text-sig", + }, + { type: "tool_call_start", contentIndex: 2, callId: "call_2", name: "lookup" }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call_2", + argumentsDelta: '{"query":', + }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call_2", + argumentsDelta: '"pi"}', + }, + { + type: "tool_call", + contentIndex: 2, + callId: "call_2", + name: "lookup", + input: { query: "pi" }, + }, + { + type: "replay_metadata", + target: "tool_call", + contentIndex: 2, + callId: "call_2", + providerId: "radius", + apiDialect: "gateway-messages", + modelId: "auto", + signature: "tool-sig", + namespace: "functions", + }, + { + type: "completed", + stopReason: "tool_use", + usage: { + inputTokens: 10, + outputTokens: 5, + cacheReadTokens: 3, + cacheWriteTokens: 2, + reasoningTokens: 1, + costUsd: 0.217, + }, + response: { + providerId: "radius", + requestedModelId: "auto", + routedModelId: "anthropic/claude-sonnet", + responseId: "response-1", + nativeStopReason: "upstream_tool_calls", + latencyMs: 42, + }, + }, + ]); +}); + +test("maps native length and aborted terminal events with safe partial state", async () => { + const request = await prepared(); + const limited = await Array.fromAsync( + decodeGatewayMessagesStream( + frames([ + { type: "text_delta", contentIndex: 0, delta: "partial" }, + { type: "done", reason: "length", usage: gatewayUsage }, + ]), + { model: gatewayModel(), request }, + ), + ); + assert.deepEqual(limited.at(-1), { + type: "completed", + stopReason: "length", + usage: { + inputTokens: 10, + outputTokens: 5, + cacheReadTokens: 3, + cacheWriteTokens: 2, + reasoningTokens: 1, + costUsd: 0.217, + }, + partial: true, + response: { + providerId: "radius", + requestedModelId: "auto", + nativeStopReason: "length", + }, + }); + + const aborted = await Array.fromAsync( + decodeGatewayMessagesStream( + frames([ + { type: "text_delta", contentIndex: 0, delta: "partial" }, + { type: "error", reason: "aborted", usage: gatewayUsage }, + ]), + { model: gatewayModel(), request }, + ), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); +}); + +test("redacts Gateway failures and preserves partial response identity", async () => { + const request = await prepared(); + const secret = "gateway-secret"; + const decoded = await Array.fromAsync( + decodeGatewayMessagesStream( + frames([ + { type: "text_delta", contentIndex: 0, delta: "partial" }, + { + type: "error", + reason: "error", + code: "upstream_rate_limit", + category: "rate_limit", + retryable: true, + errorMessage: `${secret} exhausted`, + requestedModelId: "auto", + routedModelId: "openai/gpt", + responseId: "response-2", + rawStopReason: "rate_limit", + usage: gatewayUsage, + }, + ]), + { model: gatewayModel(), request, secretValues: [secret] }, + ), + ); + assert.deepEqual(decoded.at(-1), { + type: "error", + code: "upstream_rate_limit", + message: "[REDACTED] exhausted", + retryable: true, + category: "rate_limit", + requestPhase: "streaming", + partial: true, + response: { + providerId: "radius", + requestedModelId: "auto", + routedModelId: "openai/gpt", + responseId: "response-2", + nativeStopReason: "rate_limit", + }, + }); +}); + +test("honors local cancellation before accepting a Gateway terminal", async () => { + const controller = new AbortController(); + const request = await prepared({ ...baseRequest, signal: controller.signal }); + async function* cancelling(): AsyncGenerator { + yield { data: JSON.stringify({ type: "text_delta", contentIndex: 0, delta: "partial" }) }; + controller.abort(); + yield { data: JSON.stringify({ type: "done", reason: "stop", usage: gatewayUsage }) }; + } + const decoded = await Array.fromAsync( + decodeGatewayMessagesStream(cancelling(), { model: gatewayModel(), request }), + ); + assert.deepEqual(decoded.at(-1), { type: "aborted", partial: true }); +}); + +test("fails malformed events and normalizes truncated streams exactly once", async () => { + const request = await prepared(); + await assert.rejects( + Array.fromAsync( + decodeGatewayMessagesStream( + frames([{ type: "toolcall_delta", contentIndex: 0, delta: "{}" }]), + { + model: gatewayModel(), + request, + }, + ), + ), + GatewayMessagesCodecError, + ); + await assert.rejects( + Array.fromAsync( + decodeGatewayMessagesStream( + frames([{ type: "done", reason: "stop", usage: { ...gatewayUsage, input: -1 } }]), + { model: gatewayModel(), request }, + ), + ), + /usage input must be a non-negative safe integer/, + ); + + const normalized = await Array.fromAsync( + normalizeModelStream( + decodeGatewayMessagesStream( + frames([{ type: "text_delta", contentIndex: 0, delta: "partial" }]), + { model: gatewayModel(), request }, + ), + ), + ); + assert.deepEqual(normalized.at(-1), { + type: "error", + code: "provider_stream_truncated", + message: "provider ended the stream without a terminal event", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + partial: true, + }); + assert.equal(normalized.filter((event) => event.type === "error").length, 1); +}); + +test("rejects unprepared requests and unsupported Gateway history", async () => { + const model = gatewayModel(); + assert.throws( + () => encodeGatewayMessagesRequest(model, baseRequest as never), + /requires a prepared model request/, + ); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const unsupported = await prepared({ + ...baseRequest, + messages: [ + { + role: "assistant", + continuation: { ...identity, itemId: "item-1" }, + content: [{ type: "text", text: "answer" }], + }, + ], + }); + assert.throws( + () => encodeGatewayMessagesRequest(model, unsupported), + /unsupported continuation metadata/, + ); +}); diff --git a/packages/ai/test/google-generative-ai.test.ts b/packages/ai/test/google-generative-ai.test.ts new file mode 100644 index 0000000..dd534c8 --- /dev/null +++ b/packages/ai/test/google-generative-ai.test.ts @@ -0,0 +1,679 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import type { JsonObject } from "@axl/protocol"; + +import { + decodeGoogleGenerativeAiStream, + encodeGoogleGenerativeAiRequest, + getStaticModelCatalog, + type ModelInfo, + type ModelRequest, + normalizeModelStream, + prepareModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function googleModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "google", + modelId: "gemini-3-fixture", + displayName: "Gemini Fixture", + apiDialect: "google-generative-ai", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { + minimal: "minimal", + low: "low", + medium: "medium", + high: "high", + }, + contextWindow: 1_000_000, + maxOutputTokens: 65_536, + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.1, + cacheWriteUsdPerMTok: 1.25, + }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], + }, + sampling: { + supported: ["temperature", "topP", "topK", "seed"], + customFields: ["stopSequences"], + }, + compatibility: { dialect: "google-generative-ai", supportsStrictTools: true }, + ...overrides, + }; +} + +const baseRequest: ModelRequest = { + modelId: "gemini-3-fixture", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], +}; + +async function prepared(request: ModelRequest = baseRequest, model = googleModel()) { + return prepareModelRequest(model, request); +} + +async function* frames(events: readonly unknown[]): AsyncGenerator { + for (const event of events) { + yield { event: "message", data: JSON.stringify(event) }; + } +} + +async function decode( + events: readonly unknown[], + request?: Awaited>, + model = googleModel(), +) { + return Array.fromAsync( + decodeGoogleGenerativeAiStream(frames(events), { + model, + request: request ?? (await prepared(baseRequest, model)), + }), + ); +} + +test("marks Gemini 3 models with explicit strict tool compatibility", () => { + const catalog = getStaticModelCatalog("google"); + assert.ok(catalog.length > 0); + for (const model of catalog) { + assert.equal(model.compatibility?.dialect, "google-generative-ai"); + assert.equal( + model.compatibility?.dialect === "google-generative-ai" + ? model.compatibility.supportsStrictTools === true + : false, + model.modelId.startsWith("gemini-3"), + ); + } +}); + +test("encodes prepared Google history, verified images, replay, tools, safety, cache, and controls", async () => { + const model = googleModel(); + const bytes = new Uint8Array([1, 2, 3, 4]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const request: ModelRequest = { + modelId: model.modelId, + system: "Use evidence.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: bytes.length } }, + ], + }, + { + role: "assistant", + content: [ + { type: "thinking", text: "checked", signature: { ...identity, value: "dGhpbms=" } }, + { type: "text", text: "calling", signature: { ...identity, value: "dGV4dA==" } }, + ], + toolCalls: [ + { + callId: "call-1", + name: "lookup", + input: { key: "a" }, + signature: { ...identity, value: "dG9vbA==" }, + }, + ], + }, + { + role: "tool", + callId: "call-1", + name: "lookup", + content: [ + { type: "text", text: "value" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: bytes.length } }, + ], + isError: false, + }, + { role: "user", content: [{ type: "text", text: "continue" }] }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { key: { type: "string" }, optional: { type: "number" } }, + required: ["key"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "medium", + maxOutputTokens: 40, + toolChoice: "required", + sampling: { + temperature: 0.2, + topP: 0.8, + topK: 20, + seed: 7, + custom: { stopSequences: ["END"] }, + }, + cache: { retention: "short", sessionId: "cachedContents/session-fixture" }, + safetySettings: [{ category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_ONLY_HIGH" }], + readBlob: async () => bytes, + }; + + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, request as never), + /requires a prepared model request/, + ); + const encoded = encodeGoogleGenerativeAiRequest(model, await prepared(request, model)); + assert.deepEqual(encoded.headers, { + accept: "text/event-stream", + "content-type": "application/json", + }); + assert.equal(encoded.modelId, model.modelId); + assert.equal(encoded.body.cachedContent, "cachedContents/session-fixture"); + assert.deepEqual(encoded.body.safetySettings, request.safetySettings); + assert.deepEqual(encoded.body.systemInstruction, { parts: [{ text: "Use evidence." }] }); + assert.deepEqual(encoded.body.generationConfig, { + maxOutputTokens: 40, + temperature: 0.2, + topP: 0.8, + topK: 20, + seed: 7, + stopSequences: ["END"], + thinkingConfig: { includeThoughts: true, thinkingLevel: "MEDIUM" }, + }); + assert.deepEqual(encoded.body.toolConfig, { functionCallingConfig: { mode: "ANY" } }); + const tools = encoded.body.tools as JsonObject[]; + const declaration = (tools[0]?.functionDeclarations as JsonObject[])[0]; + assert.deepEqual(declaration?.parametersJsonSchema, { + type: "object", + properties: { + key: { type: "string" }, + optional: { anyOf: [{ type: "number" }, { type: "null" }] }, + }, + required: ["key", "optional"], + additionalProperties: false, + }); + + const contents = encoded.body.contents as JsonObject[]; + assert.deepEqual(contents[0], { + role: "user", + parts: [{ text: "inspect" }, { inlineData: { mimeType: "image/png", data: "AQIDBA==" } }], + }); + assert.deepEqual(contents[1], { + role: "model", + parts: [ + { thought: true, text: "checked", thoughtSignature: "dGhpbms=" }, + { text: "calling", thoughtSignature: "dGV4dA==" }, + { + functionCall: { name: "lookup", args: { key: "a" }, id: "call-1" }, + thoughtSignature: "dG9vbA==", + }, + ], + }); + const functionResponse = (contents[2]?.parts as JsonObject[])[0]?.functionResponse as JsonObject; + assert.equal(functionResponse.id, "call-1"); + assert.deepEqual(functionResponse.response, { output: "value" }); + assert.deepEqual(functionResponse.parts, [ + { inlineData: { mimeType: "image/png", data: "AQIDBA==" } }, + ]); + assert.doesNotMatch(JSON.stringify(encoded), /api[_-]?key|authorization|credential/i); +}); + +test("encodes Gemini 2 token budgets, separate tool images, and disabled thinking", async () => { + const model = googleModel({ + modelId: "gemini-2.5-flash", + thinkingLevelMap: { minimal: "1024", low: "2048", medium: "8192", high: "24576" }, + compatibility: { dialect: "google-generative-ai" }, + }); + const enabled = await prepared( + { + modelId: model.modelId, + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + thinkingLevel: "low", + thinkingBudgets: { low: 1234 }, + maxOutputTokens: 100, + }, + model, + ); + assert.deepEqual(encodeGoogleGenerativeAiRequest(model, enabled).body.generationConfig, { + maxOutputTokens: 1334, + thinkingConfig: { includeThoughts: true, thinkingBudget: 310 }, + }); + + const disabled = await prepared( + { + modelId: model.modelId, + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + thinkingLevel: "off", + }, + model, + ); + assert.deepEqual(encodeGoogleGenerativeAiRequest(model, disabled).body.generationConfig, { + thinkingConfig: { thinkingBudget: 0 }, + }); + + const bytes = new Uint8Array([5, 6]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const toolImage = await prepared( + { + modelId: model.modelId, + messages: [ + { role: "user", content: [{ type: "text", text: "inspect" }] }, + { + role: "assistant", + content: [{ type: "text", text: "reading" }], + toolCalls: [{ callId: "call-1", name: "read", input: {} }], + }, + { + role: "tool", + callId: "call-1", + name: "read", + content: [{ type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: 2 } }], + isError: false, + }, + ], + tools: [{ name: "read", description: "Read", inputSchema: { type: "object" } }], + readBlob: async () => bytes, + }, + model, + ); + const contents = encodeGoogleGenerativeAiRequest(model, toolImage).body.contents as JsonObject[]; + assert.equal(contents.length, 4); + assert.deepEqual(contents[3], { + role: "user", + parts: [ + { text: "Tool result image:" }, + { inlineData: { mimeType: "image/png", data: "BQY=" } }, + ], + }); +}); + +test("uses minimum hidden thinking levels when Gemini 3 cannot disable thinking", async () => { + for (const [modelId, thinkingLevel] of [ + ["gemini-3.1-pro-preview", "LOW"], + ["gemini-3-flash-preview", "MINIMAL"], + ["gemma-4-26b", "MINIMAL"], + ] as const) { + const model = googleModel({ modelId }); + const request = await prepared( + { + modelId, + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + thinkingLevel: "off", + }, + model, + ); + assert.deepEqual(encodeGoogleGenerativeAiRequest(model, request).body.generationConfig, { + thinkingConfig: { thinkingLevel }, + }); + } +}); + +test("rejects malformed and unsupported Google request behavior", async () => { + const model = googleModel(); + await assert.rejects( + prepareModelRequest(model, { + ...baseRequest, + safetySettings: [ + { category: "HARM_CATEGORY_HARASSMENT", threshold: "BLOCK_ONLY_HIGH" }, + { category: "HARM_CATEGORY_HARASSMENT", threshold: "OFF" }, + ], + }), + /category is duplicated/, + ); + await assert.rejects( + prepareModelRequest(googleModel({ compatibility: { dialect: "google-generative-ai" } }), { + ...baseRequest, + tools: [ + { + name: "strict", + description: "Strict", + inputSchema: { type: "object" }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + }), + /strict schemas unsupported/, + ); + const metadata = await prepared({ ...baseRequest, metadata: { tenant: "safe" } }); + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, metadata), + /request metadata is unsupported/, + ); + const invalidCache = await prepared({ ...baseRequest, cache: { sessionId: "not-a-resource" } }); + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, invalidCache), + /cachedContents resource name/, + ); + + const bytes = new Uint8Array([9]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const unsupportedImage = await prepared({ + modelId: model.modelId, + messages: [ + { + role: "user", + content: [ + { type: "blob", blob: { sha256, mediaType: "image/bmp", sizeBytes: bytes.length } }, + ], + }, + ], + readBlob: async () => bytes, + }); + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, unsupportedImage), + /does not support image media type image\/bmp/, + ); + + const invalidSignature = await prepared({ + modelId: model.modelId, + messages: [ + { + role: "assistant", + content: [ + { + type: "text", + text: "signed", + signature: { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + value: "not-base64!", + }, + }, + ], + }, + ], + }); + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, invalidSignature), + /not a valid Google thought signature/, + ); +}); + +test("decodes interleaved Google output, replay signatures, tools, usage, cost, and identity", async () => { + const request = await prepared({ + ...baseRequest, + tools: [ + { + name: "lookup", + description: "Lookup", + inputSchema: { type: "object", properties: { key: { type: "string" } } }, + }, + ], + }); + const events = await decode( + [ + { + responseId: "resp-google", + modelVersion: "gemini-routed", + candidates: [ + { + content: { + parts: [ + { thought: true, text: "plan", thoughtSignature: "dGhpbms=" }, + { text: "answer", thoughtSignature: "dGV4dA==" }, + { + functionCall: { id: "call-1", name: "lookup", args: { key: "a" } }, + thoughtSignature: "dG9vbA==", + }, + ], + }, + }, + ], + }, + { + candidates: [{ finishReason: "STOP" }], + usageMetadata: { + promptTokenCount: 20, + cachedContentTokenCount: 5, + candidatesTokenCount: 7, + thoughtsTokenCount: 3, + totalTokenCount: 30, + }, + }, + ], + request, + ); + + assert.deepEqual(events.slice(0, 3), [ + { type: "thinking_delta", text: "plan", contentIndex: 0 }, + { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "google", + apiDialect: "google-generative-ai", + modelId: "gemini-3-fixture", + signature: "dGhpbms=", + }, + { type: "text_delta", text: "answer", contentIndex: 1 }, + ]); + assert.deepEqual(events.slice(3, 7), [ + { + type: "replay_metadata", + target: "text", + contentIndex: 1, + providerId: "google", + apiDialect: "google-generative-ai", + modelId: "gemini-3-fixture", + signature: "dGV4dA==", + }, + { type: "tool_call_start", contentIndex: 2, callId: "call-1", name: "lookup" }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call-1", + argumentsDelta: '{"key":"a"}', + }, + { + type: "tool_call", + contentIndex: 2, + callId: "call-1", + name: "lookup", + input: { key: "a" }, + }, + ]); + assert.equal(events[7]?.type, "replay_metadata"); + assert.equal(events[7]?.type === "replay_metadata" ? events[7].signature : undefined, "dG9vbA=="); + const terminal = events.at(-1); + assert.equal(terminal?.type, "completed"); + if (terminal?.type !== "completed") assert.fail("expected completion"); + assert.equal(terminal.stopReason, "tool_use"); + assert.deepEqual(terminal.usage, { + inputTokens: 15, + outputTokens: 10, + cacheReadTokens: 5, + cacheWriteTokens: 0, + reasoningTokens: 3, + costUsd: 35.5 / 1_000_000, + }); + assert.deepEqual(terminal.response, { + providerId: "google", + requestedModelId: "gemini-3-fixture", + routedModelId: "gemini-routed", + responseId: "resp-google", + nativeStopReason: "STOP", + }); +}); + +test("maps output limits and deterministic generated tool call identifiers", async () => { + const events = await decode([ + { + candidates: [ + { + content: { + parts: [ + { functionCall: { name: "first", args: {} } }, + { functionCall: { name: "second", args: {} } }, + ], + }, + finishReason: "MAX_TOKENS", + }, + ], + usageMetadata: { promptTokenCount: 1, candidatesTokenCount: 1 }, + }, + ]); + const calls = events.filter((event) => event.type === "tool_call"); + assert.deepEqual( + calls.map((event) => event.callId), + ["google_call_1", "google_call_2"], + ); + const terminal = events.at(-1); + assert.equal(terminal?.type, "completed"); + assert.equal(terminal?.type === "completed" ? terminal.stopReason : undefined, "length"); + assert.equal(terminal?.type === "completed" ? terminal.partial : undefined, true); + assert.equal( + terminal?.type === "completed" ? terminal.response?.nativeStopReason : undefined, + "MAX_TOKENS", + ); +}); + +test("maps prompt and candidate safety failures with safe partial behavior", async () => { + const blockedPrompt = await decode([ + { + promptFeedback: { + blockReason: "SAFETY", + blockReasonMessage: "unsafe prompt secret-value", + }, + }, + ]); + assert.equal(blockedPrompt[0]?.type, "error"); + + const request = await prepared(); + const candidateEvents = Array.fromAsync( + decodeGoogleGenerativeAiStream( + frames([ + { + candidates: [ + { + content: { parts: [{ text: "partial" }] }, + finishReason: "PROHIBITED_CONTENT", + finishMessage: "blocked secret-value", + }, + ], + }, + ]), + { model: googleModel(), request, secretValues: ["secret-value"] }, + ), + ); + const candidate = await candidateEvents; + const terminal = candidate.at(-1); + assert.equal(terminal?.type, "error"); + if (terminal?.type !== "error") assert.fail("expected safety error"); + assert.equal(terminal.code, "prohibited_content"); + assert.equal(terminal.partial, true); + assert.doesNotMatch(terminal.message, /secret-value/); + assert.equal(terminal.response?.nativeStopReason, "PROHIBITED_CONTENT"); +}); + +test("handles provider errors, cancellation, malformed usage, and safe partial output", async () => { + const request = await prepared(); + const providerEvents = await Array.fromAsync( + decodeGoogleGenerativeAiStream( + frames([{ error: { code: 429, status: "RESOURCE_EXHAUSTED", message: "retry secret" } }]), + { model: googleModel(), request, secretValues: ["secret"] }, + ), + ); + assert.deepEqual(providerEvents[0], { + type: "error", + code: "RESOURCE_EXHAUSTED", + message: "retry [REDACTED]", + retryable: true, + response: { providerId: "google", requestedModelId: "gemini-3-fixture" }, + }); + + const controller = new AbortController(); + async function* abortingFrames(): AsyncGenerator { + yield { + event: "message", + data: JSON.stringify({ candidates: [{ content: { parts: [{ text: "partial" }] } }] }), + }; + controller.abort(); + yield { event: "message", data: JSON.stringify({ candidates: [{ finishReason: "STOP" }] }) }; + } + const abortedRequest = await prepared({ ...baseRequest, signal: controller.signal }); + const aborted = await Array.fromAsync( + decodeGoogleGenerativeAiStream(abortingFrames(), { + model: googleModel(), + request: abortedRequest, + }), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); + + await assert.rejects( + decode([ + { + candidates: [{ finishReason: "STOP" }], + usageMetadata: { promptTokenCount: 1, cachedContentTokenCount: 2 }, + }, + ]), + /cached input tokens exceed prompt tokens/, + ); +}); + +test("ignores unknown events and normalization supplies one terminal for malformed or truncated streams", async () => { + const request = await prepared(); + async function* unknownThenStop(): AsyncGenerator { + yield { event: "future_google_event", data: "not-json" }; + yield { + event: "message", + data: JSON.stringify({ candidates: [{ finishReason: "STOP" }] }), + }; + yield { + event: "message", + data: JSON.stringify({ candidates: [{ finishReason: "MAX_TOKENS" }] }), + }; + } + const exact = await Array.fromAsync( + normalizeModelStream( + decodeGoogleGenerativeAiStream(unknownThenStop(), { model: googleModel(), request }), + ), + ); + assert.equal(exact.filter((event) => event.type === "completed").length, 1); + assert.equal(exact.at(-1)?.type, "completed"); + + const truncated = await Array.fromAsync( + normalizeModelStream( + decodeGoogleGenerativeAiStream( + frames([{ candidates: [{ content: { parts: [{ text: "partial" }] } }] }]), + { model: googleModel(), request }, + ), + ), + ); + assert.deepEqual(truncated.at(-1), { + type: "error", + code: "provider_stream_truncated", + message: "provider ended the stream without a terminal event", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + partial: true, + }); + + async function* malformed(): AsyncGenerator { + yield { event: "message", data: "{" }; + } + const malformedEvents = await Array.fromAsync( + normalizeModelStream( + decodeGoogleGenerativeAiStream(malformed(), { model: googleModel(), request }), + ), + ); + assert.equal(malformedEvents.length, 1); + assert.equal(malformedEvents[0]?.type, "error"); + assert.equal( + malformedEvents[0]?.type === "error" ? malformedEvents[0].code : undefined, + "provider_stream_failure", + ); +}); diff --git a/packages/ai/test/google-vertex.test.ts b/packages/ai/test/google-vertex.test.ts new file mode 100644 index 0000000..4340475 --- /dev/null +++ b/packages/ai/test/google-vertex.test.ts @@ -0,0 +1,272 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + decodeGoogleVertexStream, + encodeGoogleGenerativeAiRequest, + encodeGoogleVertexRequest, + getStaticModelCatalog, + googleVertexOAuthScope, + googleVertexStreamUrl, + type ModelInfo, + normalizeModelStream, + prepareModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function vertexModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "google-vertex", + modelId: "gemini-3-fixture", + displayName: "Vertex Gemini Fixture", + apiDialect: "google-vertex", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { minimal: "minimal", low: "low", medium: "medium", high: "high" }, + contextWindow: 1_000_000, + maxOutputTokens: 65_536, + cost: { inputUsdPerMTok: 1, outputUsdPerMTok: 2 }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], + }, + sampling: { supported: ["temperature", "topP", "topK", "seed"] }, + compatibility: { dialect: "google-vertex", supportsStrictTools: true }, + ...overrides, + }; +} + +async function prepared(model = vertexModel()) { + return prepareModelRequest(model, { + modelId: model.modelId, + system: "Be concise.", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { key: { type: "string" } }, + required: ["key"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "medium", + toolChoice: "required", + sampling: { temperature: 0.2, topP: 0.8, topK: 20, seed: 7 }, + }); +} + +async function* frames(events: readonly unknown[]): AsyncGenerator { + for (const event of events) yield { event: "message", data: JSON.stringify(event) }; +} + +test("marks Vertex Gemini 3 models with strict Google tool compatibility", () => { + const catalog = getStaticModelCatalog("google-vertex"); + assert.ok(catalog.length > 0); + const gemini3 = catalog.filter((model) => model.modelId.startsWith("gemini-3")); + assert.ok(gemini3.length > 0); + assert.equal( + gemini3.every( + (model) => + model.compatibility?.dialect === "google-vertex" && + model.compatibility.supportsStrictTools === true, + ), + true, + ); + assert.equal( + catalog.every((model) => model.compatibility?.dialect === "google-vertex"), + true, + ); +}); + +test("reuses Google request conversion while preserving dialect boundaries", async () => { + const model = vertexModel(); + const request = await prepared(model); + const encoded = encodeGoogleVertexRequest(model, request, { + credential: { type: "api_key", apiKey: "fixture-api-key" }, + }); + + assert.equal( + encoded.url, + "https://aiplatform.googleapis.com/v1/publishers/google/models/gemini-3-fixture:streamGenerateContent?alt=sse", + ); + assert.deepEqual(encoded.headers, { + accept: "text/event-stream", + "content-type": "application/json", + "x-goog-api-key": "fixture-api-key", + }); + assert.deepEqual(encoded.body.systemInstruction, { parts: [{ text: "Be concise." }] }); + assert.deepEqual(encoded.body.generationConfig, { + temperature: 0.2, + topP: 0.8, + topK: 20, + seed: 7, + thinkingConfig: { includeThoughts: true, thinkingLevel: "MEDIUM" }, + }); + assert.deepEqual(encoded.body.toolConfig, { functionCallingConfig: { mode: "ANY" } }); + assert.doesNotMatch(JSON.stringify(encoded.body), /fixture-api-key/); + assert.doesNotMatch(encoded.url, /fixture-api-key/); + assert.throws( + () => encodeGoogleGenerativeAiRequest(model, request), + /does not use the google-generative-ai dialect/, + ); +}); + +test("composes regional ADC and service-account requests without exposing credential files", async () => { + const model = vertexModel({ modelId: "anthropic/claude-sonnet-fixture@default" }); + const request = await prepared(model); + const adc = encodeGoogleVertexRequest(model, request, { + credential: { type: "adc", accessToken: "adc-access-token" }, + project: "fixture-project", + location: "us-central1", + }); + assert.equal( + adc.url, + "https://us-central1-aiplatform.googleapis.com/v1/projects/fixture-project/locations/us-central1/publishers/anthropic/models/claude-sonnet-fixture@default:streamGenerateContent?alt=sse", + ); + assert.equal(adc.headers.authorization, "Bearer adc-access-token"); + + const serviceAccount = encodeGoogleVertexRequest(model, request, { + credential: { + type: "service_account", + accessToken: "service-account-token", + credentialsFile: "/private/service-account.json", + }, + project: "fixture-project", + location: "eu", + }); + assert.match(serviceAccount.url, /^https:\/\/aiplatform\.eu\.rep\.googleapis\.com\/v1\//); + assert.equal(serviceAccount.headers.authorization, "Bearer service-account-token"); + assert.doesNotMatch(JSON.stringify(serviceAccount), /service-account\.json/); + assert.equal(googleVertexOAuthScope(), "https://www.googleapis.com/auth/cloud-platform"); +}); + +test("supports global and custom collection endpoints with explicit API versions", () => { + assert.equal( + googleVertexStreamUrl("gemini-3-fixture", { + credential: { type: "adc", accessToken: "token" }, + project: "fixture-project", + location: "global", + }), + "https://aiplatform.googleapis.com/v1/projects/fixture-project/locations/global/publishers/google/models/gemini-3-fixture:streamGenerateContent?alt=sse", + ); + assert.equal( + googleVertexStreamUrl("gemini-3-fixture", { + credential: { type: "adc", accessToken: "token" }, + project: "fixture-project", + location: "us-central1", + baseUrl: "https://proxy.example.com/vertex?route=primary", + apiVersion: "v1beta1", + }), + "https://proxy.example.com/vertex/v1beta1/publishers/google/models/gemini-3-fixture:streamGenerateContent?route=primary&alt=sse", + ); + assert.equal( + googleVertexStreamUrl("gemini-3-fixture", { + credential: { type: "adc", accessToken: "token" }, + project: "fixture-project", + location: "us-central1", + baseUrl: "https://proxy.example.com/v1/projects/fixture-project/locations/global", + }), + "https://proxy.example.com/v1/projects/fixture-project/locations/global/publishers/google/models/gemini-3-fixture:streamGenerateContent?alt=sse", + ); +}); + +test("fails explicitly for incomplete or malformed Vertex policy", async () => { + const model = vertexModel(); + const request = await prepared(model); + assert.throws( + () => + encodeGoogleVertexRequest(model, request, { + credential: { type: "adc", accessToken: "token" }, + location: "us-central1", + }), + /project is required/, + ); + assert.throws( + () => + encodeGoogleVertexRequest(model, request, { + credential: { type: "service_account", accessToken: "token", credentialsFile: "" }, + project: "fixture-project", + location: "us-central1", + }), + /credentials file is required/, + ); + assert.throws( + () => + encodeGoogleVertexRequest(model, request, { + credential: { type: "api_key", apiKey: "bad\nkey" }, + }), + /invalid characters/, + ); + assert.throws( + () => + encodeGoogleVertexRequest(model, request, { + credential: { type: "api_key", apiKey: "" }, + }), + /API key is a placeholder/, + ); + assert.throws( + () => + googleVertexStreamUrl("../other", { + credential: { type: "api_key", apiKey: "key" }, + }), + /model ID is invalid/, + ); +}); + +test("decodes Vertex streams with Vertex replay provenance and exact termination", async () => { + const model = vertexModel(); + const request = await prepared(model); + const events = await Array.fromAsync( + normalizeModelStream( + decodeGoogleVertexStream( + frames([ + { + responseId: "vertex-response", + modelVersion: "gemini-routed", + candidates: [ + { + content: { + parts: [{ thought: true, text: "plan", thoughtSignature: "dGhpbms=" }], + }, + }, + ], + }, + { + candidates: [{ content: { parts: [{ text: "answer" }] }, finishReason: "STOP" }], + usageMetadata: { promptTokenCount: 4, candidatesTokenCount: 2, thoughtsTokenCount: 1 }, + }, + ]), + { model, request }, + ), + ), + ); + + assert.deepEqual(events[1], { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "google-vertex", + apiDialect: "google-vertex", + modelId: "gemini-3-fixture", + signature: "dGhpbms=", + }); + const terminal = events.at(-1); + assert.equal(terminal?.type, "completed"); + if (terminal?.type !== "completed") assert.fail("expected completion"); + assert.equal(terminal.stopReason, "stop"); + assert.deepEqual(terminal.response, { + providerId: "google-vertex", + requestedModelId: "gemini-3-fixture", + routedModelId: "gemini-routed", + responseId: "vertex-response", + nativeStopReason: "STOP", + }); +}); diff --git a/packages/ai/test/mistral-conversations.test.ts b/packages/ai/test/mistral-conversations.test.ts new file mode 100644 index 0000000..5a0ab19 --- /dev/null +++ b/packages/ai/test/mistral-conversations.test.ts @@ -0,0 +1,483 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + decodeMistralConversationsStream, + encodeMistralConversationsRequest, + getStaticModelCatalog, + MistralConversationsCodecError, + type ModelInfo, + type ModelRequest, + normalizeModelStream, + prepareModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function mistralModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "mistral", + modelId: "mistral-medium-2604", + displayName: "Mistral Medium 3.5", + apiDialect: "mistral-conversations", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { + off: "none", + minimal: null, + low: null, + medium: null, + high: "high", + xhigh: null, + max: null, + }, + contextWindow: 128_000, + maxOutputTokens: 32_000, + cost: { inputUsdPerMTok: 0.4, outputUsdPerMTok: 2 }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short"], + }, + compatibility: { dialect: "mistral-conversations", supportsStrictTools: true }, + ...overrides, + }; +} + +const baseRequest: ModelRequest = { + modelId: "mistral-medium-2604", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], +}; + +async function prepared(request: ModelRequest = baseRequest, model = mistralModel()) { + return prepareModelRequest(model, request); +} + +async function* frames(items: readonly (string | Record)[]) { + for (const item of items) { + yield { data: typeof item === "string" ? item : JSON.stringify(item) } satisfies SseFrame; + } +} + +test("generated Mistral catalog declares codec capabilities explicitly", () => { + const models = getStaticModelCatalog("mistral"); + const strict = models.find((model) => model.modelId === "mistral-medium-2604"); + const ordinary = models.find((model) => model.modelId === "devstral-medium-latest"); + assert.deepEqual(strict?.compatibility, { + dialect: "mistral-conversations", + supportsStrictTools: true, + }); + assert.deepEqual(ordinary?.compatibility, { dialect: "mistral-conversations" }); +}); + +test("encodes prepared history, images, strict tools, reasoning, caching, and sampling", async () => { + const model = mistralModel(); + const image = new Uint8Array([1, 2, 3]); + const sha256 = createHash("sha256").update(image).digest("hex"); + const request = await prepared({ + modelId: model.modelId, + system: "Be precise.", + messages: [ + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: image.length } }, + ], + }, + { + role: "assistant", + content: [ + { type: "thinking", text: "reason" }, + { type: "text", text: "calling" }, + ], + toolCalls: [{ callId: "long-call-id", name: "lookup", input: { query: "pi" } }], + }, + { + role: "tool", + callId: "long-call-id", + name: "lookup", + content: [ + { type: "text", text: "failed" }, + { type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: image.length } }, + ], + isError: true, + }, + ], + tools: [ + { + name: "lookup", + description: "Look up a value", + inputSchema: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + thinkingLevel: "high", + maxOutputTokens: 100, + toolChoice: "required", + sampling: { + temperature: 0.2, + topP: 0.8, + frequencyPenalty: 0.1, + presencePenalty: 0.3, + seed: 42, + }, + cache: { retention: "short", sessionId: "session-1" }, + readBlob: async () => image, + }); + const normalizedCallId = + request.messages[1]?.role === "assistant" + ? request.messages[1].toolCalls?.[0]?.callId + : undefined; + assert.match(normalizedCallId ?? "", /^[A-Za-z0-9]{9}$/); + + const encoded = encodeMistralConversationsRequest(model, request); + assert.deepEqual(encoded.headers, { "x-affinity": "session-1" }); + assert.deepEqual(encoded.body, { + model: model.modelId, + stream: true, + messages: [ + { role: "system", content: "Be precise." }, + { + role: "user", + content: [ + { type: "text", text: "inspect" }, + { type: "image_url", image_url: "data:image/png;base64,AQID" }, + ], + }, + { + role: "assistant", + prefix: false, + content: [ + { type: "thinking", thinking: [{ type: "text", text: "reason" }] }, + { type: "text", text: "calling" }, + ], + tool_calls: [ + { + id: normalizedCallId, + type: "function", + function: { name: "lookup", arguments: '{"query":"pi"}' }, + index: 0, + }, + ], + }, + { + role: "tool", + tool_call_id: normalizedCallId, + name: "lookup", + content: [ + { type: "text", text: "[tool error] failed" }, + { type: "image_url", image_url: "data:image/png;base64,AQID" }, + ], + }, + ], + max_tokens: 100, + tools: [ + { + type: "function", + function: { + name: "lookup", + description: "Look up a value", + parameters: { + type: "object", + properties: { query: { type: "string" } }, + required: ["query"], + additionalProperties: false, + }, + strict: true, + }, + }, + ], + tool_choice: "required", + reasoning_effort: "high", + temperature: 0.2, + top_p: 0.8, + frequency_penalty: 0.1, + presence_penalty: 0.3, + random_seed: 42, + prompt_cache_key: "session-1", + }); +}); + +test("uses native prompt mode for reasoning models without effort metadata", async () => { + const { thinkingLevelMap: _thinkingLevelMap, ...baseModel } = mistralModel(); + const model: ModelInfo = { + ...baseModel, + modelId: "magistral-medium-latest", + capabilities: { toolUse: true, structuredOutput: false, imageInput: false }, + compatibility: { dialect: "mistral-conversations" }, + }; + const request = await prepared( + { + modelId: model.modelId, + messages: baseRequest.messages, + thinkingLevel: "medium", + cache: { retention: "none" }, + }, + model, + ); + const encoded = encodeMistralConversationsRequest(model, request); + assert.equal(encoded.body.prompt_mode, "reasoning"); + assert.equal(encoded.body.reasoning_effort, undefined); + assert.deepEqual(encoded.headers, {}); +}); + +test("decodes native reasoning, text, fragmented tools, usage, cost, and routed identity", async () => { + const request = await prepared(); + const decoded = await Array.fromAsync( + decodeMistralConversationsStream( + frames([ + { + id: "response-1", + model: "mistral-medium-routed", + choices: [ + { + finish_reason: null, + delta: { content: [{ type: "thinking", thinking: [{ type: "text", text: "why" }] }] }, + }, + ], + }, + { + choices: [ + { finish_reason: null, delta: { content: [{ type: "text", text: "answer" }] } }, + ], + }, + { + choices: [ + { + finish_reason: null, + delta: { + tool_calls: [ + { id: "abc123456", index: 0, function: { name: "lookup", arguments: '{"q":' } }, + ], + }, + }, + ], + }, + { + choices: [ + { + finish_reason: "tool_calls", + delta: { tool_calls: [{ index: 0, function: { name: "", arguments: '"pi"}' } }] }, + }, + ], + usage: { + prompt_tokens: 10, + completion_tokens: 4, + total_tokens: 14, + prompt_tokens_details: { cached_tokens: 3 }, + }, + }, + "[DONE]", + ]), + { model: mistralModel(), request, startedAtMs: 10, now: () => 52 }, + ), + ); + assert.deepEqual(decoded, [ + { type: "thinking_delta", text: "why", contentIndex: 0 }, + { type: "text_delta", text: "answer", contentIndex: 1 }, + { + type: "tool_call_start", + contentIndex: 2, + callId: "abc123456", + name: "lookup", + }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "abc123456", + argumentsDelta: '{"q":', + }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "abc123456", + argumentsDelta: '"pi"}', + }, + { + type: "tool_call", + contentIndex: 2, + callId: "abc123456", + name: "lookup", + input: { q: "pi" }, + }, + { + type: "completed", + stopReason: "tool_use", + usage: { + inputTokens: 7, + outputTokens: 4, + cacheReadTokens: 3, + cacheWriteTokens: 0, + reasoningTokens: 0, + costUsd: 0.0000108, + }, + response: { + providerId: "mistral", + requestedModelId: baseRequest.modelId, + routedModelId: "mistral-medium-routed", + responseId: "response-1", + nativeStopReason: "tool_calls", + latencyMs: 42, + }, + }, + ]); +}); + +test("maps native length and error stop reasons without losing partial output", async () => { + const request = await prepared(); + const limited = await Array.fromAsync( + decodeMistralConversationsStream( + frames([ + { choices: [{ finish_reason: null, delta: { content: "partial" } }] }, + { choices: [{ finish_reason: "model_length", delta: {} }] }, + ]), + { model: mistralModel(), request }, + ), + ); + assert.deepEqual(limited.at(-1), { + type: "completed", + stopReason: "length", + usage: { + inputTokens: 0, + outputTokens: 0, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + costUsd: 0, + }, + partial: true, + response: { + providerId: "mistral", + requestedModelId: baseRequest.modelId, + nativeStopReason: "model_length", + }, + }); + + const failed = await Array.fromAsync( + decodeMistralConversationsStream( + frames([ + { choices: [{ finish_reason: null, delta: { content: "partial" } }] }, + { choices: [{ finish_reason: "error", delta: {} }] }, + ]), + { model: mistralModel(), request }, + ), + ); + assert.deepEqual(failed.at(-1), { + type: "error", + code: "error", + message: "Provider stopped with: error", + retryable: false, + category: "provider_internal", + requestPhase: "streaming", + partial: true, + response: { + providerId: "mistral", + requestedModelId: baseRequest.modelId, + nativeStopReason: "error", + }, + }); +}); + +test("redacts provider failures and cancels streams with safe partial state", async () => { + const request = await prepared(); + const secret = "secret-value"; + const failed = await Array.fromAsync( + decodeMistralConversationsStream( + frames([{ error: { code: "rate_limit", message: `${secret} overloaded` } }]), + { model: mistralModel(), request, secretValues: [secret] }, + ), + ); + assert.deepEqual(failed[0], { + type: "error", + code: "rate_limit", + message: "[REDACTED] overloaded", + retryable: true, + category: "rate_limit", + requestPhase: "streaming", + response: { providerId: "mistral", requestedModelId: baseRequest.modelId }, + }); + + const controller = new AbortController(); + const cancelled = await prepared({ ...baseRequest, signal: controller.signal }); + async function* cancelling(): AsyncGenerator { + yield { + data: JSON.stringify({ choices: [{ finish_reason: null, delta: { content: "partial" } }] }), + }; + controller.abort(); + yield { data: JSON.stringify({ choices: [{ finish_reason: "stop", delta: {} }] }) }; + } + const aborted = await Array.fromAsync( + decodeMistralConversationsStream(cancelling(), { model: mistralModel(), request: cancelled }), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); +}); + +test("fails malformed frames and normalizes truncated streams exactly once", async () => { + const request = await prepared(); + await assert.rejects( + Array.fromAsync( + decodeMistralConversationsStream(frames(["not-json"]), { + model: mistralModel(), + request, + }), + ), + MistralConversationsCodecError, + ); + const normalized = await Array.fromAsync( + normalizeModelStream( + decodeMistralConversationsStream( + frames([{ choices: [{ finish_reason: null, delta: { content: "partial" } }] }]), + { model: mistralModel(), request }, + ), + ), + ); + assert.deepEqual(normalized.at(-1), { + type: "error", + code: "provider_stream_truncated", + message: "provider ended the stream without a terminal event", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + partial: true, + }); + assert.equal(normalized.filter((event) => event.type === "error").length, 1); +}); + +test("rejects unprepared requests and unsupported request data", async () => { + const model = mistralModel(); + assert.throws( + () => encodeMistralConversationsRequest(model, baseRequest as never), + /requires a prepared model request/, + ); + const metadata = await prepared({ ...baseRequest, metadata: { team: "search" } }); + assert.throws( + () => encodeMistralConversationsRequest(model, metadata), + /cannot render request metadata/, + ); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const signed = await prepared({ + ...baseRequest, + messages: [ + { + role: "assistant", + content: [{ type: "thinking", text: "private", signature: { ...identity, value: "sig" } }], + }, + ], + }); + assert.throws( + () => encodeMistralConversationsRequest(model, signed), + /unsupported replay signature/, + ); +}); diff --git a/packages/ai/test/model-contract.test.ts b/packages/ai/test/model-contract.test.ts new file mode 100644 index 0000000..e395e72 --- /dev/null +++ b/packages/ai/test/model-contract.test.ts @@ -0,0 +1,161 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import type { BlobReference } from "@axl/protocol"; + +import { + collectModelStream, + type ModelInfo, + type ModelProvider, + makeFakeModelInfo, + modelCostRates, + safeProviderDiagnostic, + safeProviderMessage, + usageCostUsd, +} from "../src/index.ts"; + +const tieredCost = { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + tiers: [ + { inputTokensAbove: 100_000, inputUsdPerMTok: 2, outputUsdPerMTok: 3 }, + { inputTokensAbove: 200_000, inputUsdPerMTok: 4, outputUsdPerMTok: 5 }, + ], +} as const; + +test("model metadata carries typed dialect policy and compatibility", () => { + const model: ModelInfo = makeFakeModelInfo({ + apiDialect: "openai-chat", + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short", "long"], + }, + endpoint: { + type: "template", + template: "https://{account}.example.test/{gateway}", + variables: [ + { name: "account", setting: "accountId", required: true }, + { name: "gateway", setting: "gatewayId", required: true }, + ], + }, + availability: { status: "preview", reason: "Limited region availability" }, + compatibility: { + dialect: "openai-chat", + supportsDeveloperRole: false, + maxTokensField: "max_tokens", + thinkingFormat: "openrouter", + thinkingTokenBudgetField: "thinking_budget_tokens", + routing: { + order: ["first", "second"], + allowFallbacks: false, + dataCollection: "deny", + }, + }, + }); + + assert.equal(model.apiDialect, model.compatibility?.dialect); + assert.equal(model.endpoint?.type, "template"); + assert.equal(model.cache?.defaultRetention, "short"); + assert.equal(model.availability?.status, "preview"); +}); + +test("request-wide pricing selects the highest matching tier", () => { + const lowUsage = { + inputTokens: 50_000, + outputTokens: 1_000, + cacheReadTokens: 0, + cacheWriteTokens: 0, + }; + const tierUsage = { + inputTokens: 90_000, + outputTokens: 1_000, + cacheReadTokens: 20_001, + cacheWriteTokens: 0, + }; + const highestUsage = { + inputTokens: 200_001, + outputTokens: 1_000, + cacheReadTokens: 0, + cacheWriteTokens: 0, + }; + + assert.equal(modelCostRates(tieredCost, lowUsage).inputUsdPerMTok, 1); + assert.equal(modelCostRates(tieredCost, tierUsage).inputUsdPerMTok, 2); + assert.equal(modelCostRates(tieredCost, highestUsage).inputUsdPerMTok, 4); + assert.equal(usageCostUsd(tieredCost, tierUsage), (2 * 90_000 + 3 * 1_000) / 1_000_000); +}); + +test("normalization rejects malformed provider events through a safe terminal", async () => { + const stream = (async function* () { + yield { type: "text_delta", text: "partial", contentIndex: -1 } as never; + })(); + const result = await collectModelStream(stream); + assert.equal(result.events.length, 1); + assert.deepEqual(result.terminal, { + type: "error", + code: "provider_stream_failure", + message: "modelStreamEvent.contentIndex must be a non-negative safe integer", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + }); +}); + +test("provider diagnostics redact known secrets and remain bounded", () => { + const secret = "fixture-provider-secret"; + assert.equal( + safeProviderMessage(`request failed for ${secret}`, [secret]), + "request failed for [REDACTED]", + ); + assert.deepEqual(safeProviderDiagnostic("request_failed", secret, "error", [secret]), { + code: "request_failed", + message: "[REDACTED]", + severity: "error", + }); + assert.equal(safeProviderMessage("x".repeat(2_500)).length, 2_000); +}); + +test("native image generation remains part of ModelProvider", async () => { + const generatedBlob: BlobReference = { + sha256: "a".repeat(64), + mediaType: "image/png", + sizeBytes: 3, + }; + const provider: ModelProvider = { + id: "image-provider", + displayName: "Image Provider", + authMethods: ["keyless"], + listModels: async () => [], + listImageModels: async () => [ + { + providerId: "image-provider", + modelId: "image-model", + displayName: "Image Model", + apiDialect: "openrouter-images", + input: ["text"], + output: ["image"], + }, + ], + stream: () => { + throw new Error("text streaming is not configured"); + }, + generateImages: async (request) => ({ + providerId: "image-provider", + requestedModelId: request.modelId, + images: [await request.writeBlob(new Uint8Array([1, 2, 3]), { mediaType: "image/png" })], + }), + }; + + const models = await provider.listImageModels?.(); + const result = await provider.generateImages?.({ + modelId: "image-model", + prompt: "draw a test", + writeBlob: async () => generatedBlob, + }); + assert.equal(models?.[0]?.apiDialect, "openrouter-images"); + assert.deepEqual(result?.images, [generatedBlob]); +}); diff --git a/packages/ai/test/models-config.test.ts b/packages/ai/test/models-config.test.ts new file mode 100644 index 0000000..c215039 --- /dev/null +++ b/packages/ai/test/models-config.test.ts @@ -0,0 +1,85 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; + +import { + getStaticModelCatalog, + InMemoryCredentialStore, + loadConfiguredProviders, +} from "../src/index.ts"; + +const context = { env: () => undefined, fileExists: async () => false }; + +test("models.json loads named providers with isolated credentials and keyless endpoints", async (t) => { + const home = await mkdtemp(join(tmpdir(), "axl-model-config-")); + t.after(() => rm(home, { recursive: true, force: true })); + const store = new InMemoryCredentialStore(); + assert.deepEqual(await loadConfiguredProviders(home, { store, context }), []); + const model = getStaticModelCatalog("deepseek")[0]; + await writeFile( + join(home, "models.json"), + JSON.stringify({ + providers: { + local: { baseUrl: "http://127.0.0.1:11434/v1", models: [model] }, + work: { + displayName: "Work proxy", + baseUrl: "https://example.com/v1", + apiKeyEnvironmentVariables: ["WORK_API_KEY"], + models: [model], + }, + }, + }), + ); + const providers = await loadConfiguredProviders(home, { store, context }); + assert.deepEqual( + providers.map((p) => p.id), + ["local", "work"], + ); + const [local, work] = providers; + assert.ok(local); + assert.ok(work?.authentication); + assert.equal((await local.listModels())[0]?.providerId, "local"); + assert.deepEqual(providers[0]?.authMethods, ["keyless"]); + await work.authentication.login("api_key", { + prompt: async () => "obviously-fake-work-key", + notify: () => {}, + }); + assert.equal((await store.read("work"))?.type, "api_key"); + assert.equal(await store.read("local"), undefined); + assert.equal(await store.read("custom"), undefined); +}); + +test("models.json rejects unsafe configuration and explicitly rejects the retired filename", async (t) => { + const home = await mkdtemp(join(tmpdir(), "axl-model-config-")); + t.after(() => rm(home, { recursive: true, force: true })); + const options = { store: new InMemoryCredentialStore(), context }; + const model = getStaticModelCatalog("deepseek")[0]; + const valid = { baseUrl: "https://example.com/v1", models: [model] }; + for (const providers of [ + { openai: valid }, + { "../escape": valid }, + { local: { ...valid, apiKey: "misplaced-secret" } }, + { local: { ...valid, baseUrl: "https://user:secret@example.com" } }, + { local: { ...valid, headers: { Authorization: "misplaced-secret" } } }, + { local: { ...valid, models: [null] } }, + { local: { ...valid, models: [{ ...model, reasoning: "yes" }] } }, + { local: { ...valid, models: [{ ...model, thinkingLevelMap: 42 }] } }, + { local: { ...valid, models: [{ ...model, thinkingLevelMap: [] }] } }, + { local: { ...valid, models: [{ ...model, maxOutputTokens: -1 }] } }, + { local: { ...valid, models: [{ ...model, compatibility: {} }] } }, + { local: { ...valid, models: [{ ...model, compatibility: [] }] } }, + ]) { + await writeFile(join(home, "models.json"), JSON.stringify({ providers })); + await assert.rejects(loadConfiguredProviders(home, options), (error: Error) => { + assert.doesNotMatch(error.message, /misplaced-secret/); + return /Invalid model configuration/.test(error.message); + }); + } + await writeFile(join(home, "custom-provider.json"), "{}"); + await assert.rejects(loadConfiguredProviders(home, options), /replaced.*models.json/); +}); diff --git a/packages/ai/test/openai-chat.test.ts b/packages/ai/test/openai-chat.test.ts new file mode 100644 index 0000000..132aab7 --- /dev/null +++ b/packages/ai/test/openai-chat.test.ts @@ -0,0 +1,688 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + decodeOpenAiChatStream, + encodeOpenAiChatRequest, + makeFakeModelInfo, + normalizeModelStream, + OpenAiChatCodecError, + prepareModelRequest, + type ModelInfo, + type ModelRequest, + type ModelStreamEvent, + type PreparedModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function chatModel(overrides: Partial = {}): ModelInfo { + return makeFakeModelInfo({ + providerId: "openrouter", + modelId: "openai/gpt-test", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { off: "none", medium: "medium", high: "high" }, + cache: { + supported: true, + defaultRetention: "none", + supportedRetentions: ["none", "short", "long"], + }, + sampling: { + supported: [ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", + ], + customFields: ["mirostat"], + }, + compatibility: { + dialect: "openai-chat", + supportsStore: true, + supportsDeveloperRole: true, + supportsReasoningEffort: true, + supportsUsageInStreaming: true, + supportsFinishReason: true, + maxTokensField: "max_completion_tokens", + requiresToolResultName: true, + thinkingFormat: "openrouter", + thinkingTokenBudgetField: "thinking_budget_tokens", + supportsGrammarTools: true, + supportsStrictTools: true, + cacheControlFormat: "anthropic", + sessionAffinityFormat: "openrouter", + supportsLongCacheRetention: true, + }, + ...overrides, + }); +} + +async function prepare( + request: Partial = {}, + model = chatModel(), +): Promise { + return prepareModelRequest(model, { + modelId: model.modelId, + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + ...request, + }); +} + +async function* frames(values: readonly unknown[]): AsyncGenerator { + for (const value of values) { + yield { data: typeof value === "string" ? value : JSON.stringify(value) }; + } +} + +async function decode( + values: readonly unknown[], + request: PreparedModelRequest, + model = chatModel(), +): Promise { + return Array.fromAsync(decodeOpenAiChatStream(frames(values), { model, request })); +} + +test("encodes every prepared Chat request control without reloading blobs", async () => { + const bytes = new Uint8Array([1, 2, 3]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + let reads = 0; + const model = chatModel(); + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const details = [{ type: "reasoning.encrypted", id: "reasoning-1", data: "opaque" }]; + const request = await prepare( + { + system: "Stable system prompt", + messages: [ + { role: "user", content: [{ type: "text", text: "inspect" }] }, + { + role: "assistant", + content: [ + { + type: "thinking", + text: "", + signature: { ...identity, value: JSON.stringify(details) }, + }, + { type: "text", text: "Using the reader." }, + ], + toolCalls: [{ callId: "call-1", name: "files.read", input: { path: "image.png" } }], + }, + { + role: "tool", + callId: "call-1", + name: "files.read", + content: [ + { type: "text", text: "image" }, + { + type: "blob", + blob: { sha256, mediaType: "image/png", sizeBytes: bytes.byteLength }, + }, + ], + isError: false, + }, + ], + tools: [ + { + name: "files.read", + description: "Read a file", + inputSchema: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + { + name: "code.expression", + description: "Evaluate an expression", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { lark: "start: NUMBER" } }, + }, + ], + thinkingLevel: "medium", + thinkingBudgets: { medium: 2_000 }, + maxOutputTokens: 3_000, + toolChoice: "auto", + sampling: { + temperature: 0.2, + topP: 0.9, + topK: 20, + minP: 0.1, + frequencyPenalty: -0.5, + presencePenalty: 0.5, + repetitionPenalty: 1.1, + seed: 7, + custom: { mirostat: 2 }, + }, + cache: { retention: "long", sessionId: "session-1" }, + readBlob: async () => { + reads += 1; + return bytes; + }, + }, + model, + ); + + assert.equal(reads, 1); + const encoded = encodeOpenAiChatRequest(model, request, "routed-model"); + assert.equal(reads, 1); + assert.equal(encoded.body.model, "routed-model"); + assert.equal(encoded.body.stream, true); + assert.deepEqual(encoded.body.stream_options, { include_usage: true }); + assert.equal(encoded.body.store, false); + assert.equal(encoded.body.max_completion_tokens, 5_000); + assert.equal(encoded.body.thinking_budget_tokens, 2_000); + assert.deepEqual(encoded.body.reasoning, { effort: "medium" }); + assert.equal(encoded.body.tool_choice, "auto"); + assert.equal(encoded.body.temperature, 0.2); + assert.equal(encoded.body.top_p, 0.9); + assert.equal(encoded.body.top_k, 20); + assert.equal(encoded.body.min_p, 0.1); + assert.equal(encoded.body.frequency_penalty, -0.5); + assert.equal(encoded.body.presence_penalty, 0.5); + assert.equal(encoded.body.repetition_penalty, 1.1); + assert.equal(encoded.body.seed, 7); + assert.equal(encoded.body.mirostat, 2); + assert.equal(encoded.body.prompt_cache_key, "session-1"); + assert.equal(encoded.body.prompt_cache_retention, "24h"); + assert.deepEqual(encoded.headers, { "x-session-id": "session-1" }); + + const messages = encoded.body.messages as Record[]; + assert.equal(messages[0]?.role, "developer"); + assert.deepEqual(messages[0]?.content, [ + { + type: "text", + text: "Stable system prompt", + cache_control: { type: "ephemeral", ttl: "1h" }, + }, + ]); + assert.deepEqual(messages[2]?.reasoning_details, details); + assert.deepEqual(messages[2]?.tool_calls, [ + { + id: "call-1", + type: "function", + function: { name: "files_read", arguments: '{"path":"image.png"}' }, + }, + ]); + assert.deepEqual(messages[3], { + role: "tool", + tool_call_id: "call-1", + name: "files_read", + content: "image", + }); + assert.deepEqual(messages[4], { + role: "user", + content: [ + { type: "text", text: "Attached image(s) from tool result:" }, + { + type: "image_url", + image_url: { url: "data:image/png;base64,AQID" }, + cache_control: { type: "ephemeral", ttl: "1h" }, + }, + ], + }); + + const tools = encoded.body.tools as Record[]; + assert.deepEqual(tools[0], { + type: "function", + function: { + name: "files_read", + description: "Read a file", + parameters: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + additionalProperties: false, + }, + strict: true, + }, + }); + assert.deepEqual(tools[1], { + type: "custom", + custom: { + name: "code_expression", + description: "Evaluate an expression", + format: { type: "grammar", grammar: { syntax: "lark", definition: "start: NUMBER" } }, + }, + cache_control: { type: "ephemeral", ttl: "1h" }, + }); +}); + +test("preserves a prepared cache marker on plain text content", async () => { + const model = chatModel(); + const request = await prepare( + { + cache: { retention: "short" }, + messages: [{ role: "user", content: [{ type: "text", text: "cache me" }] }], + }, + model, + ); + const messages = encodeOpenAiChatRequest(model, request).body.messages as Record< + string, + unknown + >[]; + assert.deepEqual(messages[0], { + role: "user", + content: [{ type: "text", text: "cache me", cache_control: { type: "ephemeral" } }], + }); +}); + +test("encodes prepared grammar history and maps provider-visible names", async () => { + const model = chatModel(); + const request = await prepare( + { + messages: [ + { role: "user", content: [{ type: "text", text: "calculate" }] }, + { + role: "assistant", + content: [{ type: "text", text: "" }], + toolCalls: [ + { callId: "call-grammar", name: "code.expression", input: { expression: "1+1" } }, + ], + }, + { + role: "tool", + callId: "call-grammar", + name: "code.expression", + content: [{ type: "text", text: "2" }], + isError: false, + }, + ], + tools: [ + { + name: "code.expression", + description: "Evaluate", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { regex: ".+" } }, + }, + ], + }, + model, + ); + const messages = encodeOpenAiChatRequest(model, request).body.messages as Record< + string, + unknown + >[]; + assert.deepEqual(messages[1]?.tool_calls, [ + { + id: "call-grammar", + type: "custom", + custom: { name: "code_expression", input: "1+1" }, + }, + ]); +}); + +test("renders every declared Chat reasoning format from prepared controls", async () => { + const cases: readonly [ + NonNullable["thinkingFormat"]>, + Record, + ][] = [ + ["openai", { reasoning_effort: "medium" }], + ["openrouter", { reasoning: { effort: "medium" } }], + ["deepseek", { thinking: { type: "enabled" }, reasoning_effort: "medium" }], + ["together", { reasoning: { enabled: true }, reasoning_effort: "medium" }], + ["zai", { thinking: { type: "enabled", clear_thinking: false }, reasoning_effort: "medium" }], + ["qwen", { enable_thinking: true, reasoning_effort: "medium" }], + ["chat-template", { chat_template_kwargs: { enable_thinking: true } }], + ["baseten", { chat_template_args: { enable_thinking: true }, reasoning_effort: "medium" }], + ["string-thinking", { thinking: "medium" }], + ["ant-ling", { reasoning: { effort: "medium" } }], + ]; + + for (const [thinkingFormat, expected] of cases) { + const model = chatModel({ + compatibility: { + dialect: "openai-chat", + thinkingFormat, + supportsReasoningEffort: true, + }, + }); + const request = await prepare({ thinkingLevel: "medium" }, model); + const body = encodeOpenAiChatRequest(model, request).body; + for (const [key, value] of Object.entries(expected)) assert.deepEqual(body[key], value); + } +}); + +test("rejects unprepared input and prepared controls Chat cannot render", async () => { + const model = chatModel(); + assert.throws( + () => encodeOpenAiChatRequest(model, { modelId: model.modelId, messages: [] } as never), + /prepared model request/, + ); + const metadata = await prepare({ metadata: { trace: "safe" } }, model); + assert.throws(() => encodeOpenAiChatRequest(model, metadata), /cannot render request metadata/); + + const identity = { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + }; + const continuation = await prepare( + { + messages: [ + { + role: "assistant", + content: [{ type: "text", text: "prior" }], + continuation: { ...identity, responseId: "response-1" }, + }, + ], + }, + model, + ); + assert.throws(() => encodeOpenAiChatRequest(model, continuation), /continuation metadata/); +}); + +test("decodes interleaved reasoning, text, tools, usage, and routed identity", async () => { + const model = chatModel({ + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.5, + cacheWriteUsdPerMTok: 0.75, + }, + }); + const request = await prepare( + { + tools: [ + { + name: "files.read", + description: "Read", + inputSchema: { type: "object" }, + }, + ], + }, + model, + ); + const events = await Array.fromAsync( + decodeOpenAiChatStream( + frames([ + { id: "chat-1", model: "routed/model", choices: [{ delta: { reasoning: "plan" } }] }, + { choices: [{ delta: { content: "answer" } }] }, + { + choices: [ + { + delta: { + tool_calls: [ + { + index: 0, + id: "call-9", + type: "function", + function: { name: "files_read", arguments: '{"path"' }, + }, + ], + }, + }, + ], + }, + { + choices: [ + { + delta: { tool_calls: [{ index: 0, function: { arguments: ':"README.md"}' } }] }, + }, + ], + }, + { choices: [{ delta: {}, finish_reason: "tool_calls" }] }, + { + choices: [], + usage: { + prompt_tokens: 120, + completion_tokens: 30, + prompt_tokens_details: { cached_tokens: 20, cache_write_tokens: 10 }, + completion_tokens_details: { reasoning_tokens: 8 }, + }, + }, + "[DONE]", + ]), + { model, request, startedAtMs: 10, now: () => 25 }, + ), + ); + + assert.deepEqual(events.slice(0, 6), [ + { type: "thinking_delta", text: "plan", contentIndex: 0 }, + { type: "text_delta", text: "answer", contentIndex: 1 }, + { type: "tool_call_start", contentIndex: 2, callId: "call-9", name: "files.read" }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call-9", + argumentsDelta: '{"path"', + }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call-9", + argumentsDelta: ':"README.md"}', + }, + { + type: "tool_call", + contentIndex: 2, + callId: "call-9", + name: "files.read", + input: { path: "README.md" }, + }, + ]); + const terminal = events[6]; + assert.equal(terminal?.type, "completed"); + if (terminal?.type !== "completed") assert.fail("expected completion"); + assert.equal(terminal.stopReason, "tool_use"); + assert.deepEqual(terminal.usage, { + inputTokens: 90, + outputTokens: 30, + cacheReadTokens: 20, + cacheWriteTokens: 10, + reasoningTokens: 8, + costUsd: 0.0001675, + }); + assert.deepEqual(terminal.response, { + providerId: "openrouter", + requestedModelId: "openai/gpt-test", + routedModelId: "routed/model", + responseId: "chat-1", + nativeStopReason: "tool_calls", + latencyMs: 15, + }); +}); + +test("decodes grammar tool input into its canonical object", async () => { + const model = chatModel(); + const request = await prepare( + { + tools: [ + { + name: "code.expression", + description: "Evaluate", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { regex: ".+" } }, + }, + ], + }, + model, + ); + const events = await decode( + [ + { + choices: [ + { + delta: { + tool_calls: [ + { + index: 0, + id: "custom-1", + type: "custom", + custom: { name: "code_expression", input: "1+" }, + }, + ], + }, + }, + ], + }, + { + choices: [{ delta: { tool_calls: [{ index: 0, custom: { input: "1" } }] } }], + }, + { choices: [{ delta: {}, finish_reason: "tool_calls" }] }, + "[DONE]", + ], + request, + model, + ); + assert.deepEqual(events.at(-2), { + type: "tool_call", + contentIndex: 0, + callId: "custom-1", + name: "code.expression", + input: { expression: "1+1" }, + }); +}); + +test("fails malformed streams and provider errors without false completion", async () => { + const model = chatModel(); + const request = await prepare({}, model); + await assert.rejects( + Array.fromAsync(decodeOpenAiChatStream(frames(["{bad json"]), { model, request })), + OpenAiChatCodecError, + ); + await assert.rejects( + decode( + [{ choices: [{ delta: { reasoning_details: [{ type: "reasoning.encrypted" }] } }] }], + request, + model, + ), + /canonical stream cannot retain/, + ); + await assert.rejects( + decode( + [ + { + choices: [ + { + delta: { + tool_calls: [ + { + index: 0, + id: "call-bad", + function: { name: "files_read", arguments: "[]" }, + }, + ], + }, + finish_reason: "tool_calls", + }, + ], + }, + "[DONE]", + ], + request, + model, + ), + /undecodable arguments/, + ); + + const failed = await Array.fromAsync( + decodeOpenAiChatStream( + frames([ + { + error: { + code: "rate_limit_exceeded", + message: "credential secret-value was rejected", + }, + }, + ]), + { model, request, secretValues: ["secret-value"] }, + ), + ); + assert.deepEqual(failed, [ + { + type: "error", + code: "rate_limit_exceeded", + message: "credential [REDACTED] was rejected", + retryable: true, + response: { providerId: "openrouter", requestedModelId: "openai/gpt-test" }, + }, + ]); +}); + +test("unknown and truncated frames normalize to one error terminal", async () => { + const model = chatModel(); + const request = await prepare({}, model); + const events = await Array.fromAsync( + normalizeModelStream( + decodeOpenAiChatStream(frames([{ vendor_extension: true }, "[DONE]"]), { + model, + request, + }), + ), + ); + assert.deepEqual(events, [ + { + type: "error", + code: "provider_stream_truncated", + message: "provider ended the stream without a terminal event", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + }, + ]); + + const partial = await Array.fromAsync( + normalizeModelStream( + decodeOpenAiChatStream(frames([{ choices: [{ delta: { content: "safe" } }] }, "{bad json"]), { + model, + request, + }), + ), + ); + assert.deepEqual(partial[0], { type: "text_delta", text: "safe", contentIndex: 0 }); + assert.deepEqual(partial[1], { + type: "error", + code: "provider_stream_failure", + message: "Provider sent an undecodable Chat stream frame", + retryable: false, + category: "stream_interrupted", + requestPhase: "streaming", + partial: true, + }); +}); + +test("cancellation terminates once and marks emitted content partial", async () => { + const controller = new AbortController(); + const model = chatModel(); + const request = await prepare({ signal: controller.signal }, model); + async function* cancellingFrames(): AsyncGenerator { + yield { data: JSON.stringify({ choices: [{ delta: { content: "partial" } }] }) }; + controller.abort(); + yield { data: JSON.stringify({ choices: [{ delta: { content: "ignored" } }] }) }; + } + const events = await Array.fromAsync( + normalizeModelStream( + decodeOpenAiChatStream(cancellingFrames(), { model, request }), + controller.signal, + ), + ); + assert.deepEqual(events, [ + { type: "text_delta", text: "partial", contentIndex: 0 }, + { type: "aborted", partial: true }, + ]); +}); diff --git a/packages/ai/test/openai-codex-responses.test.ts b/packages/ai/test/openai-codex-responses.test.ts new file mode 100644 index 0000000..61acfe3 --- /dev/null +++ b/packages/ai/test/openai-codex-responses.test.ts @@ -0,0 +1,435 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + decodeOpenAiCodexResponsesStream, + encodeOpenAiCodexResponsesRequest, + extractOpenAiCodexAccountId, + type ModelInfo, + type ModelRequest, + normalizeModelStream, + OpenAiCodexResponsesCodecError, + openAiCodexResponsesUrl, + prepareModelRequest, + type SseFrame, +} from "../src/index.ts"; + +function codexModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "openai-codex", + modelId: "gpt-5.4", + displayName: "GPT-5.4", + apiDialect: "openai-codex-responses", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { minimal: "low", xhigh: "xhigh" }, + contextWindow: 400_000, + maxOutputTokens: 128_000, + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.1, + cacheWriteUsdPerMTok: 1.25, + }, + cache: { supported: true, defaultRetention: "short", supportedRetentions: ["none", "short"] }, + sampling: { supported: ["temperature", "topP"], customFields: ["service_tier"] }, + endpoint: { type: "fixed", baseUrl: "https://chatgpt.com/backend-api/codex" }, + headers: { "x-public-client": "fixture" }, + compatibility: { + dialect: "openai-codex-responses", + supportsStrictTools: true, + supportsGrammarTools: true, + supportsMaxOutputTokens: true, + }, + ...overrides, + }; +} + +function token(accountId = "account-fixture"): string { + const payload = Buffer.from( + JSON.stringify({ "https://api.openai.com/auth": { chatgpt_account_id: accountId } }), + ).toString("base64url"); + return `header.${payload}.signature`; +} + +const baseRequest: ModelRequest = { + modelId: "gpt-5.4", + system: "Be concise.", + messages: [{ role: "user", content: [{ type: "text", text: "inspect" }] }], + tools: [ + { + name: "read_file", + description: "Read a file", + inputSchema: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + }, + }, + ], + thinkingLevel: "minimal", + maxOutputTokens: 8, + sampling: { temperature: 0.2, custom: { service_tier: "flex" } }, + cache: { retention: "short", sessionId: "session-1" }, +}; + +async function prepared(request: ModelRequest = baseRequest, model = codexModel()) { + return prepareModelRequest(model, request); +} + +async function* frames(events: readonly unknown[]): AsyncGenerator { + for (const event of events) yield { data: JSON.stringify(event) }; +} + +async function decode(events: readonly unknown[], request?: Awaited>) { + return Array.fromAsync( + decodeOpenAiCodexResponsesStream(frames(events), { + model: codexModel(), + request: request ?? (await prepared()), + }), + ); +} + +test("composes required subscription headers and Codex request policy", async () => { + const model = codexModel(); + assert.throws( + () => + encodeOpenAiCodexResponsesRequest(model, baseRequest as never, { + auth: { apiKey: token() }, + source: "fixture", + secretValues: [], + }), + /requires a prepared model request/, + ); + + const accessToken = token(); + const encoded = encodeOpenAiCodexResponsesRequest(model, await prepared(), { + auth: { + apiKey: accessToken, + headers: { Authorization: "untrusted override", "x-resolved-client": "fixture" }, + }, + source: "fixture", + secretValues: [accessToken], + }); + + assert.equal(encoded.url, "https://chatgpt.com/backend-api/codex/responses"); + assert.deepEqual(encoded.headers, { + "x-public-client": "fixture", + "x-resolved-client": "fixture", + authorization: `Bearer ${accessToken}`, + "chatgpt-account-id": "account-fixture", + originator: "axl", + "user-agent": "axl", + "openai-beta": "responses=experimental", + accept: "text/event-stream", + "content-type": "application/json", + "session-id": "session-1", + "x-client-request-id": "session-1", + }); + assert.deepEqual(encoded.body, { + model: "gpt-5.4", + input: [{ role: "user", content: [{ type: "input_text", text: "inspect" }] }], + stream: true, + store: false, + instructions: "Be concise.", + max_output_tokens: 16, + tools: [ + { + type: "function", + name: "read_file", + description: "Read a file", + parameters: { + type: "object", + properties: { path: { type: "string" } }, + required: ["path"], + }, + strict: null, + }, + ], + reasoning: { effort: "low", summary: "auto" }, + include: ["reasoning.encrypted_content"], + temperature: 0.2, + service_tier: "flex", + prompt_cache_key: "session-1", + text: { verbosity: "low" }, + tool_choice: "auto", + parallel_tool_calls: true, + }); + assert.equal(JSON.stringify(encoded.body).includes(accessToken), false); +}); + +test("validates subscription identity, endpoint policy, and cache header limits", async () => { + assert.equal(extractOpenAiCodexAccountId(token("account-2")), "account-2"); + assert.throws(() => extractOpenAiCodexAccountId("not-a-token"), OpenAiCodexResponsesCodecError); + assert.equal( + openAiCodexResponsesUrl("https://proxy.example.test/custom?route=one"), + "https://proxy.example.test/custom/codex/responses?route=one", + ); + assert.throws(() => openAiCodexResponsesUrl("not a URL"), /Invalid OpenAI Codex base URL/); + + const model = codexModel(); + const sessionId = "x".repeat(70); + const request = await prepared({ ...baseRequest, cache: { retention: "short", sessionId } }); + const encoded = encodeOpenAiCodexResponsesRequest(model, request, { + auth: { apiKey: token() }, + source: "fixture", + secretValues: [], + }); + assert.equal(encoded.headers["session-id"], "x".repeat(64)); + assert.equal(encoded.headers["x-client-request-id"], "x".repeat(64)); + assert.equal(encoded.body.prompt_cache_key, "x".repeat(64)); + assert.throws( + () => + encodeOpenAiCodexResponsesRequest(model, request, { + auth: {}, + source: "fixture", + secretValues: [], + }), + /requires a resolved subscription token/, + ); +}); + +test("replays only provenance-bound Codex continuation metadata with full stateless history", async () => { + const model = codexModel(); + const request = await prepared( + { + modelId: model.modelId, + messages: [ + { role: "user", content: [{ type: "text", text: "first" }] }, + { + role: "assistant", + content: [ + { + type: "thinking", + text: "considered", + signature: { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + value: '{"type":"reasoning","id":"rs-1","encrypted_content":"opaque"}', + }, + }, + { + type: "text", + text: "answer", + continuation: { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + responseId: "resp-1", + itemId: "msg-1", + }, + }, + ], + continuation: { + providerId: model.providerId, + apiDialect: model.apiDialect, + modelId: model.modelId, + responseId: "resp-1", + }, + }, + { role: "user", content: [{ type: "text", text: "second" }] }, + ], + }, + model, + ); + const encoded = encodeOpenAiCodexResponsesRequest(model, request, { + auth: { apiKey: token() }, + source: "fixture", + secretValues: [], + }); + assert.equal("previous_response_id" in encoded.body, false); + assert.deepEqual(encoded.body.input, [ + { role: "user", content: [{ type: "input_text", text: "first" }] }, + { type: "reasoning", id: "rs-1", encrypted_content: "opaque" }, + { + type: "message", + role: "assistant", + status: "completed", + id: "msg-1", + content: [{ type: "output_text", text: "answer", annotations: [] }], + }, + { role: "user", content: [{ type: "input_text", text: "second" }] }, + ]); +}); + +test("maps Codex response.done to canonical interleaved output and exact completion", async () => { + const events = await decode([ + { type: "codex.rate_limits", rate_limits: { allowed: true } }, + { type: "response.created", response: { id: "resp-1", model: "gpt-5.4-routed" } }, + { + type: "response.output_item.added", + output_index: 0, + item: { type: "reasoning", id: "rs-1" }, + }, + { type: "response.reasoning_summary_text.delta", output_index: 0, delta: "think" }, + { + type: "response.output_item.done", + output_index: 0, + item: { type: "reasoning", id: "rs-1", encrypted_content: "opaque" }, + }, + { + type: "response.output_item.added", + output_index: 1, + item: { type: "message", id: "msg-1" }, + }, + { type: "response.output_text.delta", output_index: 1, delta: "done" }, + { + type: "response.output_item.done", + output_index: 1, + item: { type: "message", id: "msg-1", phase: "final_answer" }, + }, + { + type: "response.done", + response: { + id: "resp-1", + model: "gpt-5.4-routed", + status: "completed", + usage: { + input_tokens: 20, + output_tokens: 5, + input_tokens_details: { cached_tokens: 10 }, + output_tokens_details: { reasoning_tokens: 2 }, + }, + }, + }, + { type: "response.output_text.delta", output_index: 1, delta: "never" }, + ]); + + assert.deepEqual( + events.map((event) => event.type), + ["thinking_delta", "replay_metadata", "text_delta", "replay_metadata", "completed"], + ); + assert.deepEqual(events.at(-1), { + type: "completed", + stopReason: "stop", + usage: { + inputTokens: 10, + outputTokens: 5, + cacheReadTokens: 10, + cacheWriteTokens: 0, + reasoningTokens: 2, + costUsd: 0.000021, + }, + response: { + providerId: "openai-codex", + requestedModelId: "gpt-5.4", + routedModelId: "gpt-5.4-routed", + responseId: "resp-1", + nativeStopReason: "completed", + }, + }); + assert.equal( + events.some((event) => JSON.stringify(event).includes(token())), + false, + ); +}); + +test("maps Codex incomplete, provider errors, and cancellation with partial output", async () => { + const incomplete = await decode([ + { + type: "response.done", + response: { + id: "resp-short", + status: "incomplete", + incomplete_details: { reason: "max_output_tokens" }, + usage: {}, + }, + }, + ]); + assert.equal(incomplete[0]?.type, "completed"); + if (incomplete[0]?.type === "completed") { + assert.equal(incomplete[0].stopReason, "length"); + assert.equal(incomplete[0].partial, true); + } + + const secret = "credential-fixture-value"; + const failed = await Array.fromAsync( + decodeOpenAiCodexResponsesStream( + frames([ + { + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: "msg-p" }, + }, + { type: "response.output_text.delta", output_index: 0, delta: "safe" }, + { type: "error", error: { code: "server_error", message: `${secret} failed` } }, + ]), + { model: codexModel(), request: await prepared(), secretValues: [secret] }, + ), + ); + const failedTerminal = failed.at(-1); + assert.equal(failedTerminal?.type, "error"); + if (failedTerminal?.type === "error") { + assert.equal(failedTerminal.partial, true); + assert.equal(failedTerminal.message.includes(secret), false); + } + + const controller = new AbortController(); + const request = await prepared({ ...baseRequest, signal: controller.signal }); + async function* cancellingFrames(): AsyncGenerator { + yield { + data: JSON.stringify({ + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: "msg-a" }, + }), + }; + yield { + data: JSON.stringify({ type: "response.output_text.delta", output_index: 0, delta: "part" }), + }; + controller.abort(); + yield { data: JSON.stringify({ type: "response.done", response: { status: "completed" } }) }; + } + const aborted = await Array.fromAsync( + decodeOpenAiCodexResponsesStream(cancellingFrames(), { model: codexModel(), request }), + ); + assert.deepEqual(aborted.at(-1), { type: "aborted", partial: true }); +}); + +test("fails malformed terminals and normalizes malformed or truncated streams exactly once", async () => { + await assert.rejects( + decode([{ type: "response.done", response: { status: "in_progress" } }]), + /no supported terminal status/, + ); + + const malformed = await Array.fromAsync( + normalizeModelStream( + decodeOpenAiCodexResponsesStream( + (async function* () { + yield { data: "{bad json" }; + })(), + { model: codexModel(), request: await prepared() }, + ), + ), + ); + assert.equal(malformed.length, 1); + assert.equal(malformed[0]?.type, "error"); + + const truncated = await Array.fromAsync( + normalizeModelStream( + decodeOpenAiCodexResponsesStream( + frames([ + { type: "vendor.future.event", detail: "ignored" }, + { + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: "msg-t" }, + }, + { type: "response.output_text.delta", output_index: 0, delta: "partial" }, + ]), + { model: codexModel(), request: await prepared() }, + ), + ), + ); + assert.deepEqual( + truncated.map((event) => event.type), + ["text_delta", "error"], + ); + const truncatedTerminal = truncated.at(-1); + assert.equal(truncatedTerminal?.type, "error"); + if (truncatedTerminal?.type === "error") assert.equal(truncatedTerminal.partial, true); +}); diff --git a/packages/ai/test/openai-responses.test.ts b/packages/ai/test/openai-responses.test.ts index 3c08d65..5bb780e 100644 --- a/packages/ai/test/openai-responses.test.ts +++ b/packages/ai/test/openai-responses.test.ts @@ -1,6 +1,5 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan // SPDX-FileCopyrightText: 2026 Kaushik Kumar -// SPDX-FileCopyrightText: 2026 Shaan Narendran // SPDX-License-Identifier: Apache-2.0 import assert from "node:assert/strict"; @@ -9,104 +8,135 @@ import test from "node:test"; import { decodeResponsesStream, encodeResponsesRequest, - makeFakeModelInfo, + type ModelInfo, type ModelRequest, type ModelStreamEvent, + normalizeModelStream, OpenAiResponsesProvider, + prepareModelRequest, ResponsesCodecError, type SseFrame, } from "../src/index.ts"; -const model = makeFakeModelInfo({ - modelId: "gpt-5", - reasoning: true, - thinkingLevelMap: { off: null, xhigh: "xhigh" }, -}); +function responsesModel(overrides: Partial = {}): ModelInfo { + return { + providerId: "openai", + modelId: "gpt-5", + displayName: "GPT-5", + apiDialect: "openai-responses", + capabilities: { toolUse: true, structuredOutput: true, imageInput: true }, + reasoning: true, + thinkingLevelMap: { off: null, xhigh: "xhigh" }, + contextWindow: 400_000, + maxOutputTokens: 128_000, + cost: { + inputUsdPerMTok: 1, + outputUsdPerMTok: 2, + cacheReadUsdPerMTok: 0.1, + cacheWriteUsdPerMTok: 1.25, + }, + cache: { supported: true, defaultRetention: "none", supportedRetentions: ["short", "long"] }, + sampling: { supported: ["temperature", "topP"], customFields: ["service_tier"] }, + compatibility: { + dialect: "openai-responses", + supportsStrictTools: true, + supportsGrammarTools: true, + supportsLongCacheRetention: true, + supportsMaxOutputTokens: true, + sessionAffinityFormat: "openai", + }, + ...overrides, + }; +} -const request: ModelRequest = { +const baseRequest: ModelRequest = { modelId: "gpt-5", system: "You are Axl.", - messages: [ - { role: "user", content: [{ type: "text", text: "run the tests" }] }, + messages: [{ role: "user", content: [{ type: "text", text: "run the tests" }] }], + tools: [ { - role: "assistant", - content: [{ type: "text", text: "Running." }], - toolCalls: [{ callId: "call-1", name: "shell", input: { command: "pnpm test" } }], - }, - { - role: "tool", - callId: "call-1", name: "shell", - content: [{ type: "text", text: "all green" }], - isError: false, + description: "Run a command", + inputSchema: { + type: "object", + properties: { command: { type: "string" } }, + required: ["command"], + }, + constraint: { type: "json-schema", strict: "require" }, }, ], - tools: [{ name: "shell", description: "Run a command", inputSchema: { type: "object" } }], thinkingLevel: "xhigh", - maxOutputTokens: 16, + maxOutputTokens: 4, + sampling: { temperature: 0.3, topP: 0.9, custom: { service_tier: "flex" } }, + cache: { retention: "long", sessionId: "session-1" }, }; -test("encodes messages, tools, thinking, and an explicit output ceiling", () => { - const body = encodeResponsesRequest(model, request, "gpt-5.6-sol"); - assert.equal(body.model, "gpt-5.6-sol"); - assert.equal(body.stream, true); - assert.equal(body.store, false); - assert.equal(body.instructions, "You are Axl."); - assert.equal(body.max_output_tokens, 16); - assert.deepEqual(body.reasoning, { effort: "xhigh" }); - assert.deepEqual(body.input, [ - { role: "user", content: [{ type: "input_text", text: "run the tests" }] }, - { role: "assistant", content: [{ type: "output_text", text: "Running." }] }, - { - type: "function_call", - call_id: "call-1", - name: "shell", - arguments: '{"command":"pnpm test"}', - }, - { type: "function_call_output", call_id: "call-1", output: "all green" }, - ]); - assert.deepEqual(body.tools, [ - { - type: "function", - name: "shell", - description: "Run a command", - parameters: { type: "object" }, - strict: false, - }, - ]); -}); +async function prepared(request: ModelRequest = baseRequest, model = responsesModel()) { + return prepareModelRequest(model, request); +} -test("off omits the reasoning parameter entirely", () => { - const body = encodeResponsesRequest( - model, - { modelId: "gpt-5", messages: [], thinkingLevel: "off" }, - "gpt-5", +async function* frames(events: readonly unknown[]): AsyncGenerator { + for (const event of events) yield { data: JSON.stringify(event) }; +} + +async function decode( + events: readonly unknown[], + request?: Awaited>, + model = responsesModel(), +): Promise { + return Array.fromAsync( + decodeResponsesStream(frames(events), { model, request: request ?? (await prepared()) }), ); - assert.equal("reasoning" in body, false); -}); +} -test("blob content fails loudly instead of being dropped", () => { - const blobRequest: ModelRequest = { - modelId: "gpt-5", - messages: [ +test("requires PreparedModelRequest and encodes prepared controls deterministically", async () => { + assert.throws( + () => encodeResponsesRequest(responsesModel(), baseRequest as never), + /requires a prepared model request/, + ); + const request = await prepared(); + const encoded = encodeResponsesRequest(responsesModel(), request, "deployment-gpt-5"); + assert.deepEqual(encoded.headers, { + session_id: "session-1", + "x-client-request-id": "session-1", + "x-session-affinity": "session-1", + }); + assert.deepEqual(encoded.body, { + model: "deployment-gpt-5", + input: [{ role: "user", content: [{ type: "input_text", text: "run the tests" }] }], + stream: true, + store: false, + instructions: "You are Axl.", + max_output_tokens: 16, + tools: [ { - role: "user", - content: [ - { type: "blob", blob: { sha256: "a".repeat(64), mediaType: "image/png", sizeBytes: 1 } }, - ], + type: "function", + name: "shell", + description: "Run a command", + parameters: { + type: "object", + properties: { command: { type: "string" } }, + required: ["command"], + additionalProperties: false, + }, + strict: true, }, ], - }; - assert.throws( - () => encodeResponsesRequest(model, blobRequest, "gpt-5"), - (error) => error instanceof ResponsesCodecError && /media transport/.test(error.message), - ); + reasoning: { effort: "xhigh", summary: "auto" }, + include: ["reasoning.encrypted_content"], + temperature: 0.3, + top_p: 0.9, + service_tier: "flex", + prompt_cache_retention: "24h", + prompt_cache_key: "session-1", + }); }); -test("encodes resolved image blobs as Responses API image parts", () => { - const digest = "a".repeat(64); - const body = encodeResponsesRequest( - model, +test("encodes verified images, grammar tools, and same-model replay metadata", async () => { + const bytes = new TextEncoder().encode("abc"); + const digest = "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad"; + const model = responsesModel(); + const request = await prepared( { modelId: "gpt-5", messages: [ @@ -117,111 +147,403 @@ test("encodes resolved image blobs as Responses API image parts", () => { { type: "blob", blob: { sha256: digest, mediaType: "image/png", sizeBytes: 3 } }, ], }, + { + role: "assistant", + content: [ + { + type: "thinking", + text: "considered", + signature: { + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + value: '{"type":"reasoning","id":"rs-1","encrypted_content":"opaque"}', + }, + }, + { + type: "text", + text: "Using a query.", + continuation: { + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + responseId: "resp-1", + itemId: "msg-1", + }, + }, + ], + toolCalls: [ + { + callId: "call-1", + name: "query", + input: { expression: "x + 1" }, + continuation: { + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + responseId: "resp-1", + itemId: "ctc-1", + namespace: "dynamic", + }, + }, + ], + }, + { + role: "tool", + callId: "call-1", + name: "query", + content: [{ type: "text", text: "2" }], + isError: false, + }, + ], + tools: [ + { + name: "query", + description: "Evaluate", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { lark: "start: /.+/" } }, + }, ], + readBlob: () => Promise.resolve(bytes), }, - "gpt-5", - new Map([[digest, "YWJj"]]), + model, ); + const body = encodeResponsesRequest(model, request).body; + assert.deepEqual(body.tools, [ + { + type: "custom", + name: "query", + description: "Evaluate", + format: { type: "grammar", syntax: "lark", definition: "start: /.+/" }, + }, + ]); assert.deepEqual(body.input, [ { role: "user", content: [ { type: "input_text", text: "inspect" }, - { - type: "input_image", - detail: "auto", - image_url: "data:image/png;base64,YWJj", - }, + { type: "input_image", detail: "auto", image_url: "data:image/png;base64,YWJj" }, ], }, + { type: "reasoning", id: "rs-1", encrypted_content: "opaque" }, + { + type: "message", + role: "assistant", + status: "completed", + id: "msg-1", + content: [{ type: "output_text", text: "Using a query.", annotations: [] }], + }, + { + type: "custom_tool_call", + call_id: "call-1", + name: "query", + input: "x + 1", + id: "ctc-1", + namespace: "dynamic", + }, + { type: "custom_tool_call_output", call_id: "call-1", output: "2" }, ]); }); -async function* frames(events: readonly unknown[]): AsyncGenerator { - for (const event of events) yield { data: JSON.stringify(event) }; -} - -async function decode(events: readonly unknown[]): Promise { - return Array.fromAsync(decodeResponsesStream(frames(events))); -} - -test("decodes a full transcript into canonical events", async () => { +test("decodes interleaved blocks, replay metadata, usage, cost, and exact completion", async () => { const events = await decode([ - { type: "response.created" }, - { type: "response.reasoning_text.delta", delta: "thinking..." }, - { type: "response.output_text.delta", delta: "Hello" }, + { type: "response.created", response: { id: "resp-1", model: "gpt-5-routed" } }, { type: "response.output_item.added", + output_index: 0, + item: { type: "reasoning", id: "rs-1" }, + }, + { type: "response.reasoning_summary_text.delta", output_index: 0, delta: "think" }, + { + type: "response.output_item.done", + output_index: 0, + item: { type: "reasoning", id: "rs-1", encrypted_content: "opaque", summary: [] }, + }, + { type: "response.output_item.added", output_index: 1, item: { type: "message", id: "msg-1" } }, + { type: "response.output_text.delta", output_index: 1, delta: "Hello" }, + { + type: "response.output_item.done", output_index: 1, - item: { type: "function_call", call_id: "call-9", name: "shell" }, + item: { type: "message", id: "msg-1", phase: "final_answer", content: [] }, + }, + { + type: "response.output_item.added", + output_index: 2, + item: { type: "function_call", id: "fc-1", call_id: "call-9", name: "shell" }, + }, + { type: "response.function_call_arguments.delta", output_index: 2, delta: '{"command":"ls"}' }, + { + type: "response.output_item.done", + output_index: 2, + item: { + type: "function_call", + id: "fc-1", + call_id: "call-9", + name: "shell", + arguments: '{"command":"ls"}', + namespace: "dynamic", + }, }, - { type: "response.function_call_arguments.delta", output_index: 1, delta: '{"command"' }, - { type: "response.function_call_arguments.delta", output_index: 1, delta: ':"ls"}' }, - { type: "response.output_item.done", output_index: 1, item: { type: "function_call" } }, { type: "response.completed", response: { + id: "resp-1", + model: "gpt-5-routed", + status: "completed", usage: { input_tokens: 120, output_tokens: 30, - input_tokens_details: { cached_tokens: 100 }, + input_tokens_details: { cached_tokens: 100, cache_write_tokens: 5 }, output_tokens_details: { reasoning_tokens: 8 }, }, }, }, - { type: "response.output_text.delta", delta: "never seen" }, + { type: "response.output_text.delta", output_index: 1, delta: "never" }, ]); assert.deepEqual(events, [ - { type: "thinking_delta", text: "thinking..." }, - { type: "text_delta", text: "Hello" }, - { type: "tool_call", callId: "call-9", name: "shell", input: { command: "ls" } }, + { type: "thinking_delta", text: "think", contentIndex: 0 }, + { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + signature: '{"type":"reasoning","id":"rs-1","encrypted_content":"opaque","summary":[]}', + responseId: "resp-1", + itemId: "rs-1", + }, + { type: "text_delta", text: "Hello", contentIndex: 1 }, + { + type: "replay_metadata", + target: "text", + contentIndex: 1, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + responseId: "resp-1", + itemId: "msg-1", + namespace: "final_answer", + }, + { type: "tool_call_start", contentIndex: 2, callId: "call-9", name: "shell" }, + { + type: "tool_call_delta", + contentIndex: 2, + callId: "call-9", + argumentsDelta: '{"command":"ls"}', + }, + { + type: "tool_call", + contentIndex: 2, + callId: "call-9", + name: "shell", + input: { command: "ls" }, + }, + { + type: "replay_metadata", + target: "tool_call", + contentIndex: 2, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + callId: "call-9", + responseId: "resp-1", + itemId: "fc-1", + namespace: "dynamic", + }, { type: "completed", stopReason: "tool_use", usage: { - inputTokens: 20, // cached tokens subtracted + inputTokens: 15, outputTokens: 30, cacheReadTokens: 100, - cacheWriteTokens: 0, + cacheWriteTokens: 5, reasoningTokens: 8, + costUsd: 0.00009125, + }, + response: { + providerId: "openai", + requestedModelId: "gpt-5", + routedModelId: "gpt-5-routed", + responseId: "resp-1", + nativeStopReason: "completed", }, }, ]); }); -test("incomplete responses complete with stopReason length", async () => { - const events = await decode([ - { type: "response.output_text.delta", delta: "truncat" }, - { type: "response.incomplete", response: { usage: { input_tokens: 5, output_tokens: 2 } } }, - ]); - assert.equal(events[1]?.type === "completed" && events[1].stopReason, "length"); +test("decodes custom tools and retains their item namespace", async () => { + const model = responsesModel(); + const request = await prepared( + { + modelId: "gpt-5", + messages: [{ role: "user", content: [{ type: "text", text: "query" }] }], + tools: [ + { + name: "query", + description: "Query", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { regex: ".+" } }, + }, + ], + }, + model, + ); + const events = await decode( + [ + { type: "response.created", response: { id: "resp-2" } }, + { + type: "response.output_item.added", + output_index: 0, + item: { type: "custom_tool_call", id: "ctc-2", call_id: "call-2", name: "query" }, + }, + { type: "response.custom_tool_call_input.delta", output_index: 0, delta: "x" }, + { + type: "response.output_item.done", + output_index: 0, + item: { + type: "custom_tool_call", + id: "ctc-2", + call_id: "call-2", + name: "query", + input: "x + 1", + namespace: "loaded", + }, + }, + { type: "response.completed", response: { id: "resp-2", status: "completed" } }, + ], + request, + model, + ); + assert.deepEqual(events.at(-3), { + type: "tool_call", + contentIndex: 0, + callId: "call-2", + name: "query", + input: { expression: "x + 1" }, + }); + assert.deepEqual(events.at(-2), { + type: "replay_metadata", + target: "tool_call", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + callId: "call-2", + responseId: "resp-2", + itemId: "ctc-2", + namespace: "loaded", + }); }); -test("failures decode to error terminals", async () => { - const failed = await decode([ - { - type: "response.failed", - response: { error: { code: "rate_limited", message: "slow down" } }, +test("maps incomplete, provider failure, cancellation, and truncation terminals", async () => { + const request = await prepared(); + const incomplete = await decode( + [ + { type: "response.created", response: { id: "resp-partial" } }, + { + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: "msg-p" }, + }, + { type: "response.output_text.delta", output_index: 0, delta: "partial" }, + { + type: "response.output_item.done", + output_index: 0, + item: { type: "message", id: "msg-p" }, + }, + { + type: "response.incomplete", + response: { + id: "resp-partial", + status: "incomplete", + incomplete_details: { reason: "max_output_tokens" }, + }, + }, + ], + request, + ); + assert.equal(incomplete.at(-1)?.type, "completed"); + assert.deepEqual(incomplete.at(-1), { + type: "completed", + stopReason: "length", + usage: { + inputTokens: 0, + outputTokens: 0, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + costUsd: 0, }, - ]); - assert.deepEqual(failed, [ - { - type: "error", - code: "rate_limited", - message: "slow down", - retryable: true, - category: "rate_limit", - requestPhase: "streaming", + partial: true, + response: { + providerId: "openai", + requestedModelId: "gpt-5", + responseId: "resp-partial", + nativeStopReason: "max_output_tokens", }, - ]); + }); + + const failure = await decode( + [ + { + type: "response.failed", + response: { + id: "resp-f", + status: "failed", + error: { code: "server_error", message: "boom" }, + }, + }, + ], + request, + ); + assert.equal(failure[0]?.type, "error"); + if (failure[0]?.type === "error") { + assert.equal(failure[0].retryable, true); + assert.equal(failure[0].category, "overloaded"); + assert.equal(failure[0].requestPhase, "streaming"); + } + + const controller = new AbortController(); + controller.abort(); + const abortedRequest = await prepared({ ...baseRequest, signal: controller.signal }); + assert.deepEqual( + await decode([{ type: "response.created", response: { id: "unused" } }], abortedRequest), + [{ type: "aborted" }], + ); - const wireError = await decode([{ type: "error", code: "bad_request", message: "no" }]); - assert.equal(wireError[0]?.type, "error"); + const truncated = await Array.fromAsync( + normalizeModelStream( + decodeResponsesStream( + frames([ + { + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: "m" }, + }, + { type: "response.output_text.delta", output_index: 0, delta: "cut" }, + ]), + { model: responsesModel(), request }, + ), + ), + ); + const truncatedTerminal = truncated.at(-1); + assert.equal(truncatedTerminal?.type, "error"); + if (truncatedTerminal?.type === "error") assert.equal(truncatedTerminal.partial, true); }); -test("classifies HTTP throttling and preserves Retry-After", async () => { - const provider = new OpenAiResponsesProvider({ +function responsesProvider(fetchImpl: typeof fetch): OpenAiResponsesProvider { + return new OpenAiResponsesProvider({ id: "responses", displayName: "Responses", authMethods: ["keyless"], @@ -230,14 +552,17 @@ test("classifies HTTP throttling and preserves Retry-After", async () => { headers: () => ({}), deploymentFor: (modelId) => modelId, }, - models: [model], + models: [responsesModel()], resolveAuth: () => Promise.resolve({ auth: {}, source: "test", secretValues: [] }), - fetch: () => - Promise.resolve(new Response("busy", { status: 429, headers: { "retry-after": "2" } })), + fetch: fetchImpl, }); +} - const events = await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })); - assert.deepEqual(events, [ +test("classifies HTTP throttling and preserves retry guidance", async () => { + const provider = responsesProvider(() => + Promise.resolve(new Response("busy", { status: 429, headers: { "retry-after": "2" } })), + ); + assert.deepEqual(await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })), [ { type: "error", code: "http_429", @@ -251,20 +576,7 @@ test("classifies HTTP throttling and preserves Retry-After", async () => { }); test("fails closed on an empty successful response", async () => { - const provider = new OpenAiResponsesProvider({ - id: "responses", - displayName: "Responses", - authMethods: ["keyless"], - endpoint: { - url: () => "https://example.test/responses", - headers: () => ({}), - deploymentFor: (modelId) => modelId, - }, - models: [model], - resolveAuth: () => Promise.resolve({ auth: {}, source: "test", secretValues: [] }), - fetch: () => Promise.resolve(new Response(null, { status: 200 })), - }); - + const provider = responsesProvider(() => Promise.resolve(new Response(null, { status: 200 }))); assert.deepEqual(await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })), [ { type: "error", @@ -277,74 +589,35 @@ test("fails closed on an empty successful response", async () => { ]); }); -test("classifies fetch failures as retryable and retains a known pre-dispatch phase", async () => { +test("retries only fetch failures known to precede dispatch", async () => { const networkCause = Object.assign(new Error("dns unavailable"), { code: "EAI_AGAIN" }); - const provider = new OpenAiResponsesProvider({ - id: "responses", - displayName: "Responses", - authMethods: ["keyless"], - endpoint: { - url: () => "https://example.test/responses", - headers: () => ({}), - deploymentFor: (modelId) => modelId, - }, - models: [model], - resolveAuth: () => Promise.resolve({ auth: {}, source: "test", secretValues: [] }), - fetch: () => Promise.reject(new TypeError("fetch failed", { cause: networkCause })), - }); - - const events = await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })); - assert.equal(events[0]?.type === "error" && events[0].retryable, true); - assert.equal(events[0]?.type === "error" && events[0].requestPhase, "before_dispatch"); -}); - -test("fails closed when fetch may have dispatched the request", async () => { - const provider = new OpenAiResponsesProvider({ - id: "responses", - displayName: "Responses", - authMethods: ["keyless"], - endpoint: { - url: () => "https://example.test/responses", - headers: () => ({}), - deploymentFor: (modelId) => modelId, - }, - models: [model], - resolveAuth: () => Promise.resolve({ auth: {}, source: "test", secretValues: [] }), - fetch: () => Promise.reject(new TypeError("fetch failed")), - }); + const safe = responsesProvider(() => + Promise.reject(new TypeError("fetch failed", { cause: networkCause })), + ); + const safeEvents = await Array.fromAsync(safe.stream({ modelId: "gpt-5", messages: [] })); + assert.equal(safeEvents[0]?.type === "error" && safeEvents[0].retryable, true); + assert.equal(safeEvents[0]?.type === "error" && safeEvents[0].requestPhase, "before_dispatch"); - const events = await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })); - assert.equal(events[0]?.type === "error" && events[0].retryable, false); - assert.equal(events[0]?.type === "error" && events[0].requestPhase, "unknown"); + const unknown = responsesProvider(() => Promise.reject(new TypeError("fetch failed"))); + const unknownEvents = await Array.fromAsync(unknown.stream({ modelId: "gpt-5", messages: [] })); + assert.equal(unknownEvents[0]?.type === "error" && unknownEvents[0].retryable, false); + assert.equal(unknownEvents[0]?.type === "error" && unknownEvents[0].requestPhase, "unknown"); }); test("classifies a terminated response stream as unsafe to redispatch", async () => { - const provider = new OpenAiResponsesProvider({ - id: "responses", - displayName: "Responses", - authMethods: ["keyless"], - endpoint: { - url: () => "https://example.test/responses", - headers: () => ({}), - deploymentFor: (modelId) => modelId, - }, - models: [model], - resolveAuth: () => Promise.resolve({ auth: {}, source: "test", secretValues: [] }), - fetch: () => - Promise.resolve( - new Response( - new ReadableStream({ - start(controller) { - controller.error(new TypeError("terminated")); - }, - }), - { status: 200 }, - ), + const provider = responsesProvider(() => + Promise.resolve( + new Response( + new ReadableStream({ + start(controller) { + controller.error(new TypeError("terminated")); + }, + }), + { status: 200 }, ), - }); - - const events = await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })); - assert.deepEqual(events, [ + ), + ); + assert.deepEqual(await Array.fromAsync(provider.stream({ modelId: "gpt-5", messages: [] })), [ { type: "error", code: "provider_stream_failed", @@ -356,50 +629,39 @@ test("classifies a terminated response stream as unsafe to redispatch", async () ]); }); -test("undecodable frames and tool arguments fail loudly", async () => { +test("rejects malformed frames, orphaned deltas, and invalid tool arguments", async () => { + const request = await prepared(); await assert.rejects( Array.fromAsync( decodeResponsesStream( (async function* () { yield { data: "{not json" }; })(), + { model: responsesModel(), request }, ), ), ResponsesCodecError, ); - const call = { - type: "response.output_item.added", - output_index: 0, - item: { type: "function_call", call_id: "c", name: "shell" }, - }; - const done = { - type: "response.output_item.done", - output_index: 0, - item: { type: "function_call" }, - }; await assert.rejects( - decode([ - call, - { type: "response.function_call_arguments.delta", output_index: 0, delta: "{broken" }, - done, - ]), - (error: unknown) => - error instanceof ResponsesCodecError && /undecodable arguments/.test(String(error)), + decode([{ type: "response.output_text.delta", output_index: 7, delta: "orphan" }], request), + /no message item/, ); await assert.rejects( - decode([ - call, - { type: "response.function_call_arguments.delta", output_index: 0, delta: "[]" }, - done, - ]), - (error: unknown) => - error instanceof ResponsesCodecError && /arguments must be an object/.test(String(error)), - ); -}); - -test("rejects an impossible output cap rather than silently exceeding it", () => { - assert.throws( - () => encodeResponsesRequest(model, { ...request, maxOutputTokens: 4 }, "gpt-5"), - /at least 16/, + decode( + [ + { + type: "response.output_item.added", + output_index: 0, + item: { type: "function_call", id: "fc", call_id: "call", name: "shell" }, + }, + { + type: "response.output_item.done", + output_index: 0, + item: { type: "function_call", id: "fc", arguments: "[]" }, + }, + ], + request, + ), + /undecodable arguments/, ); }); diff --git a/packages/ai/test/openrouter-images.test.ts b/packages/ai/test/openrouter-images.test.ts new file mode 100644 index 0000000..02a9278 --- /dev/null +++ b/packages/ai/test/openrouter-images.test.ts @@ -0,0 +1,281 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import type { BlobReference } from "@axl/protocol"; + +import { + decodeOpenRouterImageResponse, + encodeOpenRouterImageRequest, + OpenRouterImageCodecError, + type ImageGenerationRequest, + type ImageModelInfo, +} from "../src/index.ts"; + +function imageModel(overrides: Partial = {}): ImageModelInfo { + return { + providerId: "openrouter", + modelId: "google/gemini-image", + displayName: "Gemini Image", + apiDialect: "openrouter-images", + input: ["text", "image"], + output: ["image"], + cost: { inputUsdPerMTok: 1, outputUsdPerMTok: 2 }, + ...overrides, + }; +} + +function blob(bytes: Uint8Array, mediaType = "image/png"): BlobReference { + return { + sha256: createHash("sha256").update(bytes).digest("hex"), + mediaType, + sizeBytes: bytes.length, + }; +} + +function request(overrides: Partial = {}): ImageGenerationRequest { + return { + modelId: "google/gemini-image", + prompt: "Paint a quiet harbor", + writeBlob: async (bytes, metadata) => blob(bytes, metadata.mediaType), + ...overrides, + }; +} + +const pngOne = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 1]); +const pngTwo = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 2]); + +test("encodes prompt, verified image inputs, count, size, and aspect ratio", async () => { + const input = new Uint8Array([1, 2, 3]); + const reference = blob(input, "image/jpeg"); + let reads = 0; + const encoded = await encodeOpenRouterImageRequest( + imageModel(), + request({ + inputImages: [reference], + count: 2, + size: { width: 1024, height: 1024 }, + aspectRatio: "1:1", + readBlob: async (requested) => { + reads += 1; + assert.deepEqual(requested, reference); + return input; + }, + }), + ); + + assert.equal(reads, 1); + assert.deepEqual(encoded.body, { + model: "google/gemini-image", + prompt: "Paint a quiet harbor", + n: 2, + size: "1024x1024", + aspect_ratio: "1:1", + input_references: [ + { + type: "image_url", + image_url: { url: "data:image/jpeg;base64,AQID" }, + }, + ], + }); +}); + +test("encodes text-only requests without optional image fields", async () => { + const encoded = await encodeOpenRouterImageRequest(imageModel({ input: ["text"] }), request()); + assert.deepEqual(encoded.body, { + model: "google/gemini-image", + prompt: "Paint a quiet harbor", + }); +}); + +test("stores multiple images and returns usage, cost, revised prompt, and response identity", async () => { + const writes: Array<{ bytes: Uint8Array; mediaType: string }> = []; + const imageRequest = request({ + count: 2, + writeBlob: async (bytes, metadata) => { + writes.push({ bytes: new Uint8Array(bytes), mediaType: metadata.mediaType }); + return blob(bytes, metadata.mediaType); + }, + }); + const raw = { + id: "generation-1", + model: "google/gemini-image-routed", + created: 1_748_372_400, + data: [ + { + b64_json: Buffer.from(pngOne).toString("base64"), + media_type: "image/png", + revised_prompt: "A detailed quiet harbor", + }, + { + b64_json: Buffer.from(pngTwo).toString("base64"), + revised_prompt: "A detailed quiet harbor", + }, + ], + usage: { + prompt_tokens: 12, + completion_tokens: 20, + total_tokens: 32, + prompt_tokens_details: { cached_tokens: 4, cache_write_tokens: 1 }, + completion_tokens_details: { reasoning_tokens: 2 }, + cost: 0.25, + }, + }; + + const result = await decodeOpenRouterImageResponse(raw, { + model: imageModel(), + request: imageRequest, + }); + + assert.deepEqual(writes, [ + { bytes: pngOne, mediaType: "image/png" }, + { bytes: pngTwo, mediaType: "image/png" }, + ]); + assert.deepEqual(result, { + providerId: "openrouter", + requestedModelId: "google/gemini-image", + routedModelId: "google/gemini-image-routed", + responseId: "generation-1", + images: [blob(pngOne), blob(pngTwo)], + revisedPrompt: "A detailed quiet harbor", + usage: { + inputTokens: 8, + outputTokens: 20, + cacheReadTokens: 3, + cacheWriteTokens: 1, + reasoningTokens: 2, + costUsd: 0.25, + }, + }); +}); + +test("computes catalog cost only when OpenRouter omits authoritative cost", async () => { + const result = await decodeOpenRouterImageResponse( + { + data: [{ b64_json: Buffer.from(pngOne).toString("base64"), media_type: "image/png" }], + usage: { prompt_tokens: 10, completion_tokens: 5, total_tokens: 15 }, + }, + { model: imageModel(), request: request() }, + ); + assert.deepEqual(result.usage, { + inputTokens: 10, + outputTokens: 5, + cacheReadTokens: 0, + cacheWriteTokens: 0, + reasoningTokens: 0, + costUsd: 0.00002, + }); +}); + +test("redacts provider failures and preserves their retry classification", async () => { + const secret = "openrouter-secret"; + await assert.rejects( + decodeOpenRouterImageResponse( + { error: { code: "rate_limit_exceeded", message: `${secret} has no credits` } }, + { model: imageModel(), request: request(), secretValues: [secret] }, + ), + (error: unknown) => { + assert.ok(error instanceof OpenRouterImageCodecError); + assert.equal(error.code, "rate_limit_exceeded"); + assert.equal(error.message, "[REDACTED] has no credits"); + assert.equal(error.category, "rate_limit"); + assert.equal(error.retryable, true); + return true; + }, + ); +}); + +test("honors cancellation before input reads and after generated blob writes", async () => { + const before = new AbortController(); + before.abort(); + await assert.rejects( + encodeOpenRouterImageRequest(imageModel(), request({ signal: before.signal })), + (error: unknown) => error instanceof OpenRouterImageCodecError && error.aborted, + ); + + const during = new AbortController(); + await assert.rejects( + decodeOpenRouterImageResponse( + { data: [{ b64_json: Buffer.from(pngOne).toString("base64"), media_type: "image/png" }] }, + { + model: imageModel(), + request: request({ + signal: during.signal, + writeBlob: async (bytes, metadata) => { + during.abort(); + return blob(bytes, metadata.mediaType); + }, + }), + }, + ), + (error: unknown) => error instanceof OpenRouterImageCodecError && error.aborted, + ); +}); + +test("rejects malformed requests, responses, and blob boundary violations", async () => { + await assert.rejects( + encodeOpenRouterImageRequest(imageModel(), request({ count: 11 })), + /count must be an integer from 1 to 10/, + ); + await assert.rejects( + encodeOpenRouterImageRequest( + imageModel(), + request({ size: { width: 1024, height: 512 }, aspectRatio: "1:1" }), + ), + /size and aspect ratio are inconsistent/, + ); + await assert.rejects( + encodeOpenRouterImageRequest( + imageModel(), + request({ + inputImages: [blob(new Uint8Array([1]))], + readBlob: async () => new Uint8Array([2]), + }), + ), + /does not match its content address/, + ); + await assert.rejects( + decodeOpenRouterImageResponse( + { data: [{ b64_json: "not base64" }] }, + { + model: imageModel(), + request: request(), + }, + ), + /malformed base64 data/, + ); + await assert.rejects( + decodeOpenRouterImageResponse( + { data: [{ b64_json: Buffer.from(pngOne).toString("base64"), media_type: "image/png" }] }, + { + model: imageModel(), + request: request({ + writeBlob: async () => ({ + sha256: "a".repeat(64), + mediaType: "image/png", + sizeBytes: pngOne.length, + }), + }), + }, + ), + /does not match the generated bytes/, + ); + await assert.rejects( + decodeOpenRouterImageResponse({ data: [] }, { model: imageModel(), request: request() }), + /contains no images/, + ); + await assert.rejects( + encodeOpenRouterImageRequest(imageModel(), request({ prompt: "x".repeat(1024 * 1024 + 1) })), + /byte limit/, + ); + await assert.rejects( + decodeOpenRouterImageResponse( + { data: [{ b64_json: "A".repeat(8 * 1024 * 1024 + 1) }] }, + { model: imageModel(), request: request() }, + ), + /bounded base64 data/, + ); +}); diff --git a/packages/ai/test/provider-port.test.ts b/packages/ai/test/provider-port.test.ts index e80cd8d..cec6cf5 100644 --- a/packages/ai/test/provider-port.test.ts +++ b/packages/ai/test/provider-port.test.ts @@ -5,7 +5,16 @@ import assert from "node:assert/strict"; import test from "node:test"; -import { FakeModelProvider, modelPortForSession, type ModelStreamEvent } from "../src/index.ts"; +import { + FakeModelProvider, + isPreparedModelRequest, + type ModelRequest, + type ModelStreamEvent, + makeFakeModelInfo, + modelPortForRegistry, + modelPortForSession, + ProviderRegistry, +} from "../src/index.ts"; const usage = { inputTokens: 1, outputTokens: 2, cacheReadTokens: 0, cacheWriteTokens: 0 }; @@ -41,6 +50,336 @@ test("binds model choice and thinking level into kernel-shaped turns", async () assert.equal(request?.thinkingLevel, "high"); assert.equal(request?.system, "You are Axl."); assert.equal(request?.readBlob, readBlob); + assert.ok(request); + assert.equal(isPreparedModelRequest(request), true); +}); + +test("binds provider and model identity through the registry coordinator", async () => { + const provider = new FakeModelProvider({ + id: "mixed", + models: [ + { + providerId: "mixed", + modelId: "chat", + displayName: "Chat", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: false }, + reasoning: false, + contextWindow: 10_000, + maxOutputTokens: 1_000, + }, + ], + responses: [[{ type: "completed", stopReason: "stop", usage }]], + }); + const registry = new ProviderRegistry(); + registry.register(provider); + const modelPort = modelPortForRegistry(registry, { + providerId: "mixed", + modelId: "chat", + }); + + const result: ModelStreamEvent[] = []; + for await (const event of modelPort.stream({ messages: [], tools: [] })) result.push(event); + assert.equal(result.at(-1)?.type, "completed"); + assert.equal(provider.requests[0]?.modelId, "chat"); +}); + +test("retains replay metadata in assistant history for the next in-process turn", async () => { + const provider = new FakeModelProvider({ + responses: [ + [ + { type: "thinking_delta", text: "considered", contentIndex: 0 }, + { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + signature: "opaque-reasoning", + redacted: true, + responseId: "resp-1", + itemId: "rs-1", + }, + { type: "text_delta", text: "running", contentIndex: 1 }, + { + type: "replay_metadata", + target: "text", + contentIndex: 1, + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + signature: "opaque-text", + responseId: "resp-1", + itemId: "msg-1", + }, + { type: "tool_call", contentIndex: 2, callId: "call-1", name: "shell", input: {} }, + { + type: "replay_metadata", + target: "tool_call", + contentIndex: 2, + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + callId: "call-1", + signature: "opaque-tool", + responseId: "resp-1", + itemId: "fc-1", + namespace: "dynamic", + }, + { type: "completed", stopReason: "tool_use", usage }, + ], + [{ type: "completed", stopReason: "stop", usage }], + [{ type: "completed", stopReason: "stop", usage }], + ], + }); + const port = modelPortForSession(provider, { modelId: "fake-model" }); + await Array.fromAsync(port.stream({ messages: [], tools: [] })); + await Array.fromAsync( + port.stream({ + messages: [ + { + role: "assistant", + content: [ + { type: "thinking", text: "considered" }, + { type: "text", text: "running" }, + ], + toolCalls: [{ callId: "call-1", name: "shell", input: {} }], + }, + { + role: "tool", + callId: "call-1", + name: "shell", + content: [{ type: "text", text: "done" }], + isError: false, + }, + ], + tools: [{ name: "shell", description: "Run", inputSchema: { type: "object" } }], + }), + ); + + const assistant = provider.requests[1]?.messages[0]; + assert.equal(assistant?.role, "assistant"); + if (assistant?.role !== "assistant") assert.fail("expected replayed assistant message"); + assert.deepEqual(assistant.origin, { + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + }); + assert.equal( + assistant.content[0]?.type === "thinking" ? assistant.content[0].signature?.value : undefined, + "opaque-reasoning", + ); + assert.equal( + assistant.content[0]?.type === "thinking" ? assistant.content[0].redacted : undefined, + true, + ); + assert.equal( + assistant.content[1]?.type === "text" ? assistant.content[1].continuation?.itemId : undefined, + "msg-1", + ); + assert.equal( + assistant.content[1]?.type === "text" ? assistant.content[1].signature?.value : undefined, + "opaque-text", + ); + assert.equal(assistant.toolCalls?.[0]?.signature?.value, "opaque-tool"); + assert.deepEqual(assistant.toolCalls?.[0]?.continuation, { + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + responseId: "resp-1", + itemId: "fc-1", + namespace: "dynamic", + }); + assert.equal(assistant.continuation?.responseId, "resp-1"); + + await Array.fromAsync( + port.stream({ + messages: [ + { + role: "assistant", + content: [ + { type: "thinking", text: "considered" }, + { type: "text", text: "running" }, + ], + toolCalls: [{ callId: "call-1", name: "shell", input: {} }], + }, + { + role: "tool", + callId: "call-1", + name: "shell", + content: [{ type: "text", text: "done" }], + isError: false, + }, + { role: "assistant", content: [{ type: "text", text: "finished" }] }, + { role: "user", content: [{ type: "text", text: "again" }] }, + ], + tools: [{ name: "shell", description: "Run", inputSchema: { type: "object" } }], + }), + ); + const nextMessages = provider.requests[2]?.messages; + const secondAssistant = nextMessages?.filter((message) => message.role === "assistant")[1]; + assert.equal(secondAssistant?.role, "assistant"); + if (secondAssistant?.role === "assistant") { + assert.equal(secondAssistant.origin, undefined); + assert.equal(secondAssistant.continuation, undefined); + } +}); + +test("retains signature-only replay without inventing continuation state", async () => { + const provider = new FakeModelProvider({ + responses: [ + [ + { + type: "replay_metadata", + target: "text", + contentIndex: 0, + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + signature: "text-signature", + }, + { + type: "replay_metadata", + target: "tool_call", + contentIndex: 1, + callId: "call-1", + providerId: "fake", + apiDialect: "fake", + modelId: "fake-model", + signature: "tool-signature", + }, + { type: "completed", stopReason: "tool_use", usage }, + ], + [{ type: "completed", stopReason: "stop", usage }], + ], + }); + const port = modelPortForSession(provider, { modelId: "fake-model" }); + await Array.fromAsync(port.stream({ messages: [], tools: [] })); + await Array.fromAsync( + port.stream({ + messages: [ + { + role: "assistant", + content: [{ type: "text", text: "running" }], + toolCalls: [{ callId: "call-1", name: "shell", input: {} }], + }, + { + role: "tool", + callId: "call-1", + name: "shell", + content: [{ type: "text", text: "done" }], + isError: false, + }, + ], + tools: [{ name: "shell", description: "Run", inputSchema: { type: "object" } }], + }), + ); + + const assistant = provider.requests[1]?.messages[0]; + if (assistant?.role !== "assistant") assert.fail("expected replayed assistant message"); + assert.equal( + assistant.content[0]?.type === "text" ? assistant.content[0].signature?.value : undefined, + "text-signature", + ); + assert.equal( + assistant.content[0]?.type === "text" ? assistant.content[0].continuation : undefined, + undefined, + ); + assert.equal(assistant.toolCalls?.[0]?.signature?.value, "tool-signature"); + assert.equal(assistant.toolCalls?.[0]?.continuation, undefined); +}); + +test("strips foreign continuation state when a session changes providers", async () => { + const target = new FakeModelProvider({ + id: "target", + models: [ + makeFakeModelInfo({ + providerId: "target", + modelId: "target-model", + apiDialect: "openai-chat", + }), + ], + responses: [[{ type: "completed", stopReason: "stop", usage }]], + }); + const registry = new ProviderRegistry(); + registry.register(target); + const foreign = { + providerId: "source", + apiDialect: "openai-responses", + modelId: "source-model", + } as const; + const request = { + modelId: "target-model", + messages: [ + { + role: "assistant", + origin: foreign, + continuation: { ...foreign, responseId: "response-source" }, + content: [ + { + type: "thinking", + text: "portable summary", + signature: { ...foreign, value: "thinking-source" }, + }, + { + type: "text", + text: "calling", + signature: { ...foreign, value: "text-source" }, + continuation: { ...foreign, itemId: "message-source" }, + }, + ], + toolCalls: [ + { + callId: "call-source", + name: "lookup", + input: { query: "fixture" }, + signature: { ...foreign, value: "tool-source" }, + continuation: { ...foreign, itemId: "tool-source" }, + }, + ], + }, + { + role: "tool", + callId: "call-source", + name: "lookup", + content: [{ type: "text", text: "result" }], + isError: false, + }, + ], + tools: [{ name: "lookup", description: "Lookup", inputSchema: { type: "object" } }], + } as unknown as ModelRequest; + + await Array.fromAsync(registry.stream("target", request)); + + const prepared = target.requests[0]; + assert.ok(prepared); + assert.equal(isPreparedModelRequest(prepared), true); + if (!isPreparedModelRequest(prepared)) assert.fail("expected prepared target request"); + const assistant = prepared.messages[0]; + if (assistant?.role !== "assistant") assert.fail("expected prepared assistant history"); + assert.deepEqual(assistant.origin, foreign); + assert.equal(assistant.continuation, undefined); + for (const content of assistant.content) { + if (content.type === "text") { + assert.equal(content.signature, undefined); + assert.equal(content.continuation, undefined); + } + if (content.type === "thinking") assert.equal(content.signature, undefined); + } + assert.equal(assistant.toolCalls?.[0]?.signature, undefined); + assert.equal(assistant.toolCalls?.[0]?.continuation, undefined); + assert.deepEqual( + prepared.preparation.sanitizations.map((entry) => entry.reason), + [ + "foreign-provider-continuation", + "foreign-provider-signature", + "foreign-provider-signature", + "foreign-provider-continuation", + "foreign-provider-signature", + "foreign-provider-continuation", + ], + ); }); test("normalization guarantees a terminal even when the provider misbehaves", async () => { diff --git a/packages/ai/test/registry.test.ts b/packages/ai/test/registry.test.ts index cb13c7b..2a34ef5 100644 --- a/packages/ai/test/registry.test.ts +++ b/packages/ai/test/registry.test.ts @@ -1,23 +1,39 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 import assert from "node:assert/strict"; import test from "node:test"; -import { FakeModelProvider, ProviderRegistry, ProviderRegistryError } from "../src/index.ts"; +import { + type CatalogSnapshot, + collectModelStream, + FakeModelProvider, + InMemoryCatalogStore, + type ModelCatalogRefreshResult, + type ModelInfo, + type ModelProvider, + makeFakeModelInfo, + ProviderRegistry, + ProviderRegistryError, +} from "../src/index.ts"; -function makeProvider(id: string): FakeModelProvider { - return new FakeModelProvider({ id, responses: [] }); +const usage = { inputTokens: 1, outputTokens: 1, cacheReadTokens: 0, cacheWriteTokens: 0 }; + +function makeProvider(id: string, models?: readonly ModelInfo[]): FakeModelProvider { + return new FakeModelProvider({ id, ...(models === undefined ? {} : { models }), responses: [] }); } -test("registers, resolves, and lists providers", () => { +test("registers, resolves, and lists enabled providers", () => { const registry = new ProviderRegistry(); const provider = makeProvider("azure"); registry.register(provider); assert.equal(registry.get("azure"), provider); assert.equal(registry.has("azure"), true); + assert.equal(registry.isEnabled("azure"), true); assert.deepEqual(registry.list(), [provider]); + assert.deepEqual(registry.registrations(), [{ provider, enabled: true }]); }); test("rejects duplicate provider IDs and unknown lookups loudly", () => { @@ -26,11 +42,265 @@ test("rejects duplicate provider IDs and unknown lookups loudly", () => { assert.throws( () => registry.register(makeProvider("azure")), - (error) => error instanceof ProviderRegistryError && /already registered/.test(error.message), + (error) => + error instanceof ProviderRegistryError && + error.code === "provider_duplicate" && + /already registered/.test(error.message), ); assert.throws( () => registry.get("missing"), - (error) => error instanceof ProviderRegistryError && /not registered/.test(error.message), + (error) => + error instanceof ProviderRegistryError && + error.code === "provider_missing" && + /not registered/.test(error.message), + ); +}); + +test("looks models up by provider and model identity", async () => { + const registry = new ProviderRegistry(); + const first = makeProvider("first", [ + makeFakeModelInfo({ providerId: "first", modelId: "shared", apiDialect: "openai-chat" }), + ]); + const second = makeProvider("second", [ + makeFakeModelInfo({ + providerId: "second", + modelId: "shared", + apiDialect: "anthropic-messages", + }), + ]); + registry.register(first); + registry.register(second); + + assert.equal((await registry.getModel("first", "shared")).apiDialect, "openai-chat"); + assert.equal((await registry.getModel("second", "shared")).apiDialect, "anthropic-messages"); + const catalog = await registry.listModels(); + assert.deepEqual( + catalog.models.map((model) => `${model.providerId}/${model.modelId}`), + ["first/shared", "second/shared"], + ); + assert.equal(catalog.errors.size, 0); +}); + +test("dispatches mixed dialect models through their owning provider", async () => { + const models = [ + makeFakeModelInfo({ providerId: "mixed", modelId: "chat", apiDialect: "openai-chat" }), + makeFakeModelInfo({ + providerId: "mixed", + modelId: "responses", + apiDialect: "openai-responses", + }), + ]; + const provider = new FakeModelProvider({ + id: "mixed", + models, + responses: [ + [ + { type: "text_delta", text: "selected" }, + { type: "completed", stopReason: "stop", usage }, + ], + ], + }); + const registry = new ProviderRegistry(); + registry.register(provider); + + const result = await collectModelStream( + registry.stream("mixed", { modelId: "responses", messages: [] }), + ); + assert.equal(result.terminal.type, "completed"); + assert.equal(provider.requests[0]?.modelId, "responses"); + assert.equal((await registry.getModel("mixed", "responses")).apiDialect, "openai-responses"); +}); + +test("filters unavailable models and rejects their dispatch", async () => { + const registry = new ProviderRegistry(); + registry.register( + makeProvider("availability", [ + makeFakeModelInfo({ providerId: "availability", modelId: "ready" }), + makeFakeModelInfo({ + providerId: "availability", + modelId: "blocked", + availability: { status: "unavailable", reason: "region disabled" }, + }), + ]), + ); + + assert.deepEqual( + (await registry.listModels()).models.map((model) => model.modelId), + ["ready"], + ); + assert.deepEqual( + (await registry.listModels({ includeUnavailable: true })).models.map((model) => model.modelId), + ["ready", "blocked"], + ); + await assert.rejects( + registry.getModel("availability", "blocked"), + (error) => error instanceof ProviderRegistryError && error.code === "model_unavailable", + ); +}); + +test("disabled providers perform no catalog, refresh, or dispatch work", async () => { + let catalogCalls = 0; + let refreshCalls = 0; + let streamCalls = 0; + const model = makeFakeModelInfo({ providerId: "disabled" }); + const provider: ModelProvider = { + id: "disabled", + displayName: "Disabled", + authMethods: ["keyless"], + listModels: async () => { + catalogCalls += 1; + return [model]; + }, + refreshModelCatalog: async (context) => { + refreshCalls += 1; + return { + status: "updated", + providerId: context.providerId, + generation: context.generation, + source: { id: "disabled-api", kind: "provider_api" }, + models: [model], + }; + }, + stream: () => { + streamCalls += 1; + return (async function* () { + yield { type: "completed", stopReason: "stop", usage } as const; + })(); + }, + }; + const registry = new ProviderRegistry(); + registry.register(provider, { enabled: false }); + + assert.deepEqual(await registry.listModels(), { models: [], errors: new Map() }); + assert.deepEqual(await registry.refresh(), { + models: [], + errors: new Map(), + refreshedProviderIds: [], + restoredProviderIds: [], + supersededProviderIds: [], + snapshots: new Map(), + diagnostics: new Map(), + }); + await assert.rejects( + registry.getModel("disabled", model.modelId), + (error) => error instanceof ProviderRegistryError && error.code === "provider_disabled", + ); + assert.deepEqual( + { catalogCalls, refreshCalls, streamCalls }, + { catalogCalls: 0, refreshCalls: 0, streamCalls: 0 }, + ); + + registry.setEnabled("disabled", true); + assert.equal((await registry.listModels()).models.length, 1); +}); + +test("explicit refresh isolates provider failures", async () => { + const refreshed = makeFakeModelInfo({ providerId: "healthy", modelId: "new" }); + const healthy: ModelProvider = { + id: "healthy", + displayName: "Healthy", + authMethods: ["keyless"], + listModels: async () => [refreshed], + refreshModelCatalog: async (context) => ({ + status: "updated", + providerId: context.providerId, + generation: context.generation, + source: { id: "healthy-api", kind: "provider_api", revision: "catalog-7" }, + sourceUpdatedAt: 900, + etag: '"healthy-7"', + diagnostics: [{ code: "catalog.current", message: "Catalog is current", severity: "info" }], + models: [refreshed], + }), + stream: () => { + throw new Error("not used"); + }, + }; + const failed: ModelProvider = { + id: "failed", + displayName: "Failed", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async () => { + throw new Error("catalog unavailable"); + }, + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry({ now: () => 1_000 }); + registry.register(healthy); + registry.register(failed); + + const result = await registry.refresh(); + assert.deepEqual(result.models, [refreshed]); + assert.deepEqual(result.refreshedProviderIds, ["healthy"]); + assert.deepEqual(result.snapshots.get("healthy"), { + version: 1, + providerId: "healthy", + generation: 1, + checkedAt: 1_000, + updatedAt: 1_000, + sourceUpdatedAt: 900, + etag: '"healthy-7"', + source: { id: "healthy-api", kind: "provider_api", revision: "catalog-7" }, + models: [refreshed], + }); + assert.deepEqual(result.diagnostics.get("healthy"), [ + { code: "catalog.current", message: "Catalog is current", severity: "info" }, + ]); + assert.match(result.errors.get("failed")?.message ?? "", /catalog unavailable/); +}); + +test("explicit refresh honors cancellation before provider work", async () => { + let refreshCalls = 0; + const provider: ModelProvider = { + id: "cancelled", + displayName: "Cancelled", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => { + refreshCalls += 1; + return { + status: "updated", + providerId: context.providerId, + generation: context.generation, + source: { id: "cancelled-api", kind: "provider_api" }, + models: [], + }; + }, + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry(); + registry.register(provider); + const controller = new AbortController(); + controller.abort(); + + await assert.rejects(registry.refresh({ signal: controller.signal }), { name: "AbortError" }); + assert.equal(refreshCalls, 0); +}); + +test("rejects malformed provider catalogs", async () => { + const registry = new ProviderRegistry(); + registry.register( + makeProvider("wrong-owner", [makeFakeModelInfo({ providerId: "another-provider" })]), + ); + await assert.rejects( + registry.getModel("wrong-owner", "fake-model"), + (error) => error instanceof ProviderRegistryError && error.code === "catalog_failure", + ); + + const duplicates = new ProviderRegistry(); + duplicates.register( + makeProvider("duplicates", [ + makeFakeModelInfo({ providerId: "duplicates", modelId: "same" }), + makeFakeModelInfo({ providerId: "duplicates", modelId: "same" }), + ]), + ); + await assert.rejects( + duplicates.getModel("duplicates", "same"), + (error) => error instanceof ProviderRegistryError && error.code === "model_duplicate", ); }); @@ -45,6 +315,390 @@ test("disposer unregisters once, disposes the provider, and never removes a succ const second = makeProvider("azure"); registry.register(second); - await dispose(); // stale disposer must not remove the successor + await dispose(); assert.equal(registry.get("azure"), second); }); + +test("registry disposal owns every provider lifecycle and is idempotent", async () => { + const disposed: string[] = []; + const provider = (id: string): ModelProvider => ({ + id, + displayName: id, + authMethods: ["keyless"], + listModels: async () => [], + stream: () => { + throw new Error("not used"); + }, + dispose: async () => { + disposed.push(id); + }, + }); + const registry = new ProviderRegistry(); + registry.register(provider("first")); + registry.register(provider("second")); + + await registry.dispose(); + await registry.dispose(); + assert.deepEqual(disposed.sort(), ["first", "second"]); + assert.throws( + () => registry.list(), + (error) => error instanceof ProviderRegistryError && error.code === "registry_disposed", + ); +}); + +test("restores a persisted dynamic catalog before network refresh", async () => { + const store = new InMemoryCatalogStore(); + const storedModel = makeFakeModelInfo({ providerId: "dynamic", modelId: "stored" }); + await store.write("dynamic", { + version: 1, + providerId: "dynamic", + generation: 4, + checkedAt: 100, + updatedAt: 90, + source: { id: "dynamic-api", kind: "provider_api" }, + models: [storedModel], + }); + let registry!: ProviderRegistry; + let restoredBeforeRefresh = false; + let refreshCalls = 0; + const provider: ModelProvider = { + id: "dynamic", + displayName: "Dynamic", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => { + refreshCalls += 1; + restoredBeforeRefresh = + context.previous?.generation === 4 && + (await registry.getModel("dynamic", "stored")).modelId === "stored"; + return { + status: "not_modified", + providerId: context.providerId, + generation: context.generation, + source: { id: "dynamic-api", kind: "provider_api" }, + }; + }, + stream: () => { + throw new Error("not used"); + }, + }; + registry = new ProviderRegistry({ catalogStore: store, now: () => 200 }); + registry.register(provider); + + const offline = await registry.restoreCatalogs(); + assert.deepEqual(offline.restoredProviderIds, ["dynamic"]); + assert.equal(refreshCalls, 0); + assert.equal((await registry.getModel("dynamic", "stored")).modelId, "stored"); + + const result = await registry.refresh(); + assert.equal(restoredBeforeRefresh, true); + assert.deepEqual(result.restoredProviderIds, ["dynamic"]); + assert.deepEqual(result.refreshedProviderIds, ["dynamic"]); + assert.equal(result.snapshots.get("dynamic")?.checkedAt, 200); + assert.equal(result.snapshots.get("dynamic")?.generation, 4); + assert.deepEqual( + (await registry.listModels()).models.map((model) => model.modelId), + ["stored"], + ); + assert.equal((await store.read("dynamic"))?.checkedAt, 200); +}); + +test("failed and malformed refreshes retain the previous valid catalog", async () => { + const store = new InMemoryCatalogStore(); + const previous = makeFakeModelInfo({ providerId: "retained", modelId: "previous" }); + await store.write("retained", { + version: 1, + providerId: "retained", + generation: 2, + checkedAt: 100, + updatedAt: 100, + etag: '"previous"', + source: { id: "retained-api", kind: "provider_api" }, + models: [previous], + }); + let malformed = false; + const provider: ModelProvider = { + id: "retained", + displayName: "Retained", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => { + if (!malformed) throw new Error("remote unavailable"); + return { + status: "updated", + providerId: context.providerId, + generation: context.generation, + source: { id: "retained-api", kind: "provider_api" }, + models: [{ ...previous, maxOutputTokens: 0 }], + }; + }, + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register(provider); + + const failed = await registry.refresh(); + assert.match(failed.errors.get("retained")?.message ?? "", /remote unavailable/); + assert.equal((await registry.getModel("retained", "previous")).modelId, "previous"); + assert.equal((await store.read("retained"))?.etag, '"previous"'); + + malformed = true; + const invalid = await registry.refresh(); + assert.match(invalid.errors.get("retained")?.message ?? "", /invalid model catalog/); + assert.equal((await registry.getModel("retained", "previous")).modelId, "previous"); + assert.equal((await store.read("retained"))?.generation, 2); +}); + +test("cancelled and superseded refreshes cannot replace the last-known-good catalog", async () => { + const store = new InMemoryCatalogStore(); + const previous = makeFakeModelInfo({ providerId: "racing", modelId: "previous" }); + await store.write("racing", { + version: 1, + providerId: "racing", + generation: 1, + checkedAt: 10, + updatedAt: 10, + source: { id: "racing-api", kind: "provider_api" }, + models: [previous], + }); + let call = 0; + let finishFirst: ((result: ModelCatalogRefreshResult) => void) | undefined; + let finishCancelled: ((result: ModelCatalogRefreshResult) => void) | undefined; + let markStarted: (() => void) | undefined; + let markCancelledStarted: (() => void) | undefined; + const started = new Promise((resolvePromise) => { + markStarted = resolvePromise; + }); + const cancelledStarted = new Promise((resolvePromise) => { + markCancelledStarted = resolvePromise; + }); + const provider: ModelProvider = { + id: "racing", + displayName: "Racing", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: (context) => { + call += 1; + if (call === 1) { + markStarted?.(); + return new Promise((resolvePromise) => { + finishFirst = resolvePromise; + }); + } + if (call === 3) { + markCancelledStarted?.(); + return new Promise((resolvePromise) => { + finishCancelled = resolvePromise; + }); + } + return Promise.resolve({ + status: "updated", + providerId: context.providerId, + generation: context.generation, + source: { id: "racing-api", kind: "provider_api" }, + etag: '"newer"', + models: [makeFakeModelInfo({ providerId: "racing", modelId: "newer" })], + }); + }, + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register(provider); + + const first = registry.refresh(); + await started; + const second = await registry.refresh(); + const firstResult = await first; + assert.deepEqual(second.refreshedProviderIds, ["racing"]); + assert.deepEqual(firstResult.supersededProviderIds, ["racing"]); + + finishFirst?.({ + status: "updated", + providerId: "racing", + generation: 1, + source: { id: "racing-api", kind: "provider_api" }, + etag: '"older"', + models: [makeFakeModelInfo({ providerId: "racing", modelId: "older" })], + }); + await new Promise((resolvePromise) => setTimeout(resolvePromise, 0)); + assert.deepEqual( + (await registry.listModels()).models.map((model) => model.modelId), + ["newer"], + ); + assert.equal((await store.read("racing"))?.etag, '"newer"'); + + const controller = new AbortController(); + const pending = registry.refresh({ signal: controller.signal }); + await cancelledStarted; + controller.abort(); + await assert.rejects(pending, { name: "AbortError" }); + finishCancelled?.({ + status: "updated", + providerId: "racing", + generation: 3, + source: { id: "racing-api", kind: "provider_api" }, + models: [makeFakeModelInfo({ providerId: "racing", modelId: "cancelled" })], + }); + await new Promise((resolvePromise) => setTimeout(resolvePromise, 0)); + assert.deepEqual( + (await registry.listModels()).models.map((model) => model.modelId), + ["newer"], + ); + assert.equal((await store.read("racing"))?.generation, 2); +}); + +test("supersession during persistence rolls back the rejected snapshot across restart", async () => { + class SupersedingStore extends InMemoryCatalogStore { + onCommit: (() => void) | undefined; + override async write(providerId: string, snapshot: CatalogSnapshot): Promise { + await super.write(providerId, snapshot); + const callback = this.onCommit; + this.onCommit = undefined; + callback?.(); + } + } + const store = new SupersedingStore(); + const previous = makeFakeModelInfo({ providerId: "atomic", modelId: "previous" }); + await store.write("atomic", { + version: 1, + providerId: "atomic", + generation: 1, + checkedAt: 1, + updatedAt: 1, + source: { id: "atomic-api", kind: "provider_api" }, + models: [previous], + }); + const candidate = makeFakeModelInfo({ providerId: "atomic", modelId: "candidate" }); + const provider: ModelProvider = { + id: "atomic", + displayName: "Atomic", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => ({ + status: "updated", + providerId: "atomic", + generation: context.generation, + source: { id: "atomic-api", kind: "provider_api" }, + models: [candidate], + }), + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register(provider); + store.onCommit = () => { + registry.setEnabled("atomic", false); + registry.setEnabled("atomic", true); + }; + const result = await registry.refresh({ providerId: "atomic" }); + assert.deepEqual(result.supersededProviderIds, ["atomic"]); + assert.equal((await store.read("atomic"))?.models[0]?.modelId, "previous"); + + const restarted = new ProviderRegistry({ catalogStore: store }); + restarted.register(provider); + await restarted.restoreCatalogs({ providerId: "atomic" }); + assert.equal((await restarted.getModel("atomic", "previous")).modelId, "previous"); + await assert.rejects(restarted.getModel("atomic", "candidate"), /no model/); +}); + +test("restored snapshots reject unsafe provider endpoints before dispatch", async () => { + const unsafe = { + version: 1 as const, + providerId: "restored", + generation: 1, + checkedAt: 1, + updatedAt: 1, + source: { id: "restored-api", kind: "provider_api" as const }, + models: [ + { + ...makeFakeModelInfo({ providerId: "restored" }), + endpoint: { type: "fixed" as const, baseUrl: "http://169.254.169.254/latest" }, + }, + ], + }; + const store = { + read: async () => unsafe, + write: async () => undefined, + delete: async () => undefined, + }; + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register({ + id: "restored", + displayName: "Restored", + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => ({ + status: "not_modified", + providerId: "restored", + generation: context.generation, + source: unsafe.source, + }), + stream: () => { + throw new Error("must not dispatch"); + }, + }); + const result = await registry.restoreCatalogs({ providerId: "restored" }); + assert.equal(result.restoredProviderIds.length, 0); + assert.ok(result.errors.has("restored")); +}); + +test("legacy provider refresh and compatibility metadata remain source-compatible", async () => { + const model: ModelInfo = { + ...makeFakeModelInfo({ providerId: "legacy", apiDialect: "legacy-wire" }), + compatibility: { strictJsonSchema: false }, + }; + const provider: ModelProvider = { + id: "legacy", + displayName: "Legacy", + authMethods: ["keyless"], + listModels: async () => [model], + refreshModels: async () => [model], + stream: () => { + throw new Error("not used"); + }, + }; + const registry = new ProviderRegistry(); + registry.register(provider); + const result = await registry.refresh({ providerId: "legacy" }); + assert.deepEqual(result.refreshedProviderIds, ["legacy"]); + assert.equal((await registry.getModel("legacy", "fake-model")).apiDialect, "legacy-wire"); +}); + +test("dynamic refresh isolates corrupt persisted providers from healthy providers", async () => { + class IsolatedStore extends InMemoryCatalogStore { + override read(providerId: string): Promise { + if (providerId === "broken") return Promise.reject(new Error("corrupt snapshot")); + return super.read(providerId); + } + } + const source = { id: "catalog-api", kind: "provider_api" } as const; + const dynamicProvider = (id: string): ModelProvider => ({ + id, + displayName: id, + authMethods: ["keyless"], + listModels: async () => [], + refreshModelCatalog: async (context) => ({ + status: "updated", + providerId: context.providerId, + generation: context.generation, + source, + models: [makeFakeModelInfo({ providerId: id })], + }), + stream: () => { + throw new Error("not used"); + }, + }); + const registry = new ProviderRegistry({ catalogStore: new IsolatedStore() }); + registry.register(dynamicProvider("healthy")); + registry.register(dynamicProvider("broken")); + + const result = await registry.refresh(); + assert.deepEqual(result.refreshedProviderIds, ["healthy"]); + assert.match(result.errors.get("broken")?.message ?? "", /corrupt snapshot/); + assert.equal((await registry.getModel("healthy", "fake-model")).providerId, "healthy"); +}); diff --git a/packages/ai/test/remaining-providers.test.ts b/packages/ai/test/remaining-providers.test.ts new file mode 100644 index 0000000..ee9761d --- /dev/null +++ b/packages/ai/test/remaining-providers.test.ts @@ -0,0 +1,837 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + createAmazonBedrockProvider, + createAnthropicProvider, + createAzureOpenAiResponsesProvider, + createCloudflareAiGatewayProvider, + createCloudflareWorkersAiProvider, + createCustomProvider, + createGitHubCopilotProvider, + createGoogleProvider, + createGoogleVertexProvider, + createKimiCodingProvider, + createMistralProvider, + createOpenAiCodexProvider, + createOpenAiProvider, + createOpenCodeGoProvider, + createOpenCodeProvider, + createOpenRouterProvider, + createRadiusProvider, + getStaticModelCatalog, + InMemoryCatalogStore, + InMemoryCredentialStore, + login, + type ModelProvider, + parseCustomProviderConfiguration, + ProviderRegistry, +} from "../src/index.ts"; + +const ENVIRONMENT: Readonly> = { + OPENAI_API_KEY: "openai-secret", + ANTHROPIC_API_KEY: "anthropic-secret", + GEMINI_API_KEY: "google-secret", + GOOGLE_CLOUD_API_KEY: "vertex-secret", + MISTRAL_API_KEY: "mistral-secret", + KIMI_API_KEY: "kimi-secret", + OPENCODE_API_KEY: "opencode-secret", + AZURE_OPENAI_API_KEY: "azure-secret", + AZURE_OPENAI_RESOURCE_NAME: "sample-resource", + CLOUDFLARE_API_KEY: "cloudflare-secret", + CLOUDFLARE_ACCOUNT_ID: "account-one", + CLOUDFLARE_GATEWAY_ID: "gateway-one", + OPENROUTER_API_KEY: "openrouter-secret", + COPILOT_GITHUB_TOKEN: "tid=test;proxy-ep=proxy.individual.githubcopilot.com;token=copilot-secret", + RADIUS_API_KEY: "radius-secret", + AWS_BEARER_TOKEN_BEDROCK: "bedrock-secret", + AWS_REGION: "us-east-1", +}; + +const context = { + env: (name: string) => ENVIRONMENT[name], + fileExists: () => Promise.resolve(false), +}; + +async function consume(provider: ModelProvider, modelId: string): Promise { + for await (const _event of provider.stream({ modelId, messages: [] })) { + // Consume the complete normalized provider stream. + } +} + +test("dispatches every newly active static provider through its declared dialect endpoint", async () => { + const requests: { providerId: string; url: string; headers: Headers }[] = []; + const makeFetch = + (providerId: string): typeof fetch => + async (input, init) => { + requests.push({ providerId, url: String(input), headers: new Headers(init?.headers) }); + return new Response("data: [DONE]\n\n", { + status: 200, + headers: { "content-type": "text/event-stream" }, + }); + }; + const factories: readonly [string, (fetchImpl: typeof fetch) => ModelProvider][] = [ + [ + "openai", + (fetchImpl) => + createOpenAiProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + ], + [ + "anthropic", + (fetchImpl) => + createAnthropicProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "google", + (fetchImpl) => + createGoogleProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + ], + [ + "google-vertex", + (fetchImpl) => + createGoogleVertexProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "amazon-bedrock", + (fetchImpl) => + createAmazonBedrockProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "mistral", + (fetchImpl) => + createMistralProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + ], + [ + "kimi-coding", + (fetchImpl) => + createKimiCodingProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "opencode", + (fetchImpl) => + createOpenCodeProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + ], + [ + "opencode-go", + (fetchImpl) => + createOpenCodeGoProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "azure-openai-responses", + (fetchImpl) => + createAzureOpenAiResponsesProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + [ + "cloudflare-workers-ai", + (fetchImpl) => + createCloudflareWorkersAiProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + ], + ]; + + for (const [id, factory] of factories) { + const provider = factory(makeFetch(id)); + const models = await provider.listModels(); + const byDialect = new Map(models.map((model) => [model.apiDialect, model])); + for (const model of byDialect.values()) await consume(provider, model.modelId); + } + + const urls = new Map( + requests.map((request) => [ + `${request.providerId}:${new URL(request.url).pathname.split("/").at(-1)}`, + request, + ]), + ); + assert.ok(urls.has("openai:responses")); + assert.ok(urls.has("openai:completions")); + assert.ok(urls.has("anthropic:messages")); + assert.ok( + requests.some( + (request) => + request.providerId === "google" && request.url.includes(":streamGenerateContent?alt=sse"), + ), + ); + assert.ok( + requests.some( + (request) => + request.providerId === "google-vertex" && + request.url.includes(":streamGenerateContent?alt=sse"), + ), + ); + assert.ok( + requests.some( + (request) => + request.providerId === "amazon-bedrock" && + request.url.includes("bedrock-runtime.us-east-1.amazonaws.com/model/"), + ), + ); + assert.equal( + requests + .find((request) => request.providerId === "amazon-bedrock") + ?.headers.get("authorization"), + "Bearer bedrock-secret", + ); + assert.ok(urls.has("mistral:conversations")); + assert.ok(urls.has("kimi-coding:completions")); + assert.ok(urls.has("opencode:responses")); + assert.ok(urls.has("opencode:messages")); + assert.ok(urls.has("opencode:completions")); + assert.ok(urls.has("azure-openai-responses:responses")); + assert.ok( + requests.some((request) => + request.url.includes("/accounts/account-one/ai/v1/chat/completions"), + ), + ); + assert.equal( + requests.find((request) => request.providerId === "anthropic")?.headers.get("x-api-key"), + "anthropic-secret", + ); + assert.equal( + requests + .find((request) => request.providerId === "cloudflare-workers-ai") + ?.headers.get("authorization"), + "Bearer cloudflare-secret", + ); +}); + +test("dispatches Codex, Gateway, and image dialects through deterministic transports", async () => { + const codexStore = new InMemoryCredentialStore(); + const codexPayload = Buffer.from( + JSON.stringify({ "https://api.openai.com/auth": { chatgpt_account_id: "account-fixture" } }), + ).toString("base64url"); + await login(codexStore, "openai-codex", { + type: "oauth", + access: `header.${codexPayload}.signature`, + refresh: "refresh-fixture", + expiresAt: Number.MAX_SAFE_INTEGER, + }); + let codexRequest: { url: string; headers: Headers } | undefined; + const codex = createOpenAiCodexProvider({ + store: codexStore, + context, + now: () => 1_000, + fetch: async (input, init) => { + codexRequest = { url: String(input), headers: new Headers(init?.headers) }; + return new Response( + 'data: {"type":"response.done","response":{"id":"response-fixture","model":"gpt-5.4","status":"completed","usage":{}}}\n\n', + { status: 200, headers: { "content-type": "text/event-stream" } }, + ); + }, + }); + const codexModel = (await codex.listModels())[0]; + assert.ok(codexModel); + await consume(codex, codexModel.modelId); + assert.equal(codexModel.apiDialect, "openai-codex-responses"); + assert.match(codexRequest?.url ?? "", /\/codex\/responses$/); + assert.equal(codexRequest?.headers.get("chatgpt-account-id"), "account-fixture"); + + const radiusRequests: string[] = []; + const radius = createRadiusProvider({ + store: new InMemoryCredentialStore(), + context, + baseUrl: "https://radius.pi.dev", + fetch: async (input) => { + const url = String(input); + radiusRequests.push(url); + if (url.endsWith("/v1/config")) { + return Response.json({ + baseUrl: "https://radius.pi.dev/v1", + models: [ + { + id: "auto", + name: "Auto", + reasoning: true, + toolUse: true, + input: ["text"], + cost: { input: 0, output: 0 }, + contextWindow: 128_000, + maxTokens: 16_000, + }, + ], + }); + } + return new Response( + [ + 'data: {"type":"start"}', + 'data: {"type":"text_start","contentIndex":0}', + 'data: {"type":"text_delta","contentIndex":0,"delta":"ok"}', + 'data: {"type":"text_end","contentIndex":0,"content":"ok"}', + 'data: {"type":"done","reason":"stop","usage":{"input":1,"output":1,"cacheRead":0,"cacheWrite":0,"reasoning":0,"totalTokens":2,"cost":{"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"total":0}}}', + "", + ].join("\n\n"), + { status: 200, headers: { "content-type": "text/event-stream" } }, + ); + }, + }); + const radiusRegistry = new ProviderRegistry({ catalogStore: new InMemoryCatalogStore() }); + radiusRegistry.register(radius); + await radiusRegistry.refresh({ providerId: "radius" }); + const radiusEvents = await Array.fromAsync( + radiusRegistry.stream("radius", { modelId: "auto", messages: [] }), + ); + assert.equal((await radiusRegistry.getModel("radius", "auto")).apiDialect, "gateway-messages"); + assert.equal(radiusEvents.at(-1)?.type, "completed", JSON.stringify(radiusEvents)); + assert.deepEqual(radiusRequests, [ + "https://radius.pi.dev/v1/config", + "https://radius.pi.dev/v1/messages", + ]); + + const imageBytes = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]); + const openRouterRequests: string[] = []; + const openRouter = createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async (input) => { + const url = String(input); + openRouterRequests.push(url); + if (url.endsWith("/models")) { + return Response.json({ + data: [ + { + id: "image-fixture", + name: "Image Fixture", + architecture: { input_modalities: ["text"], output_modalities: ["image"] }, + context_length: 1_000, + top_provider: { max_completion_tokens: 100 }, + supported_parameters: [], + }, + ], + }); + } + return Response.json({ + id: "generation-fixture", + data: [ + { + b64_json: Buffer.from(imageBytes).toString("base64"), + media_type: "image/png", + }, + ], + }); + }, + }); + const openRouterRegistry = new ProviderRegistry({ catalogStore: new InMemoryCatalogStore() }); + openRouterRegistry.register(openRouter); + await openRouterRegistry.refresh({ providerId: "openrouter" }); + const imageModel = (await openRouterRegistry.listImageModels("openrouter"))[0]; + assert.ok(imageModel); + assert.equal(imageModel.apiDialect, "openrouter-images"); + const imageResult = await openRouter.generateImages?.({ + modelId: imageModel.modelId, + prompt: "deterministic fixture", + writeBlob: async (bytes, metadata) => ({ + sha256: createHash("sha256").update(bytes).digest("hex"), + sizeBytes: bytes.byteLength, + mediaType: metadata.mediaType, + }), + }); + assert.equal(imageResult?.images.length, 1); + assert.deepEqual(openRouterRequests, [ + "https://openrouter.ai/api/v1/models", + "https://openrouter.ai/api/v1/images", + ]); +}); + +test("validates first-party custom provider configuration", () => { + const source = getStaticModelCatalog("deepseek")[0]; + if (source === undefined) throw new Error("DeepSeek catalog is empty"); + const parsed = parseCustomProviderConfiguration({ + baseUrl: "http://127.0.0.1:11434/v1", + models: [{ ...source, providerId: "foreign" }], + apiKeyEnvironmentVariables: ["CUSTOM_API_KEY"], + }); + assert.equal(parsed.models[0]?.providerId, "custom"); + assert.throws(() => + parseCustomProviderConfiguration({ + baseUrl: "https://169.254.169.254/v1", + models: [source], + }), + ); + assert.throws(() => + parseCustomProviderConfiguration({ + baseUrl: "https://example.com/v1", + models: [source], + apiKeyEnvironmentVariables: ["not-valid"], + }), + ); +}); + +test("dispatches a keyless configured endpoint with only validated custom headers", async () => { + const source = getStaticModelCatalog("deepseek")[0]; + if (source === undefined) throw new Error("DeepSeek catalog is empty"); + let requestUrl = ""; + let headers = new Headers(); + const provider = createCustomProvider({ + store: new InMemoryCredentialStore(), + context, + baseUrl: "http://127.0.0.1:11434/v1", + headers: { "x-tenant": "local" }, + models: [{ ...source, providerId: "custom", modelId: "local-model" }], + fetch: async (input, init) => { + requestUrl = String(input); + headers = new Headers(init?.headers); + return new Response("data: [DONE]\n\n", { status: 200 }); + }, + }); + await consume(provider, "local-model"); + assert.equal(requestUrl, "http://127.0.0.1:11434/v1/chat/completions"); + assert.equal(headers.get("x-tenant"), "local"); + assert.equal(headers.has("authorization"), false); +}); + +test("dispatches a configured Responses endpoint with explicit API key authentication", async () => { + const source = getStaticModelCatalog("openai").find( + (model) => model.apiDialect === "openai-responses", + ); + if (source === undefined) throw new Error("OpenAI Responses catalog is empty"); + let requestUrl = ""; + let headers = new Headers(); + const provider = createCustomProvider({ + store: new InMemoryCredentialStore(), + context: { + env: (name) => (name === "CUSTOM_API_KEY" ? "custom-secret" : undefined), + fileExists: () => Promise.resolve(false), + }, + baseUrl: "http://127.0.0.1:11435/v1", + headers: { "x-tenant": "configured" }, + apiKeyEnvironmentVariables: ["CUSTOM_API_KEY"], + models: [{ ...source, providerId: "custom", modelId: "local-responses" }], + fetch: async (input, init) => { + requestUrl = String(input); + headers = new Headers(init?.headers); + return new Response( + 'data: {"type":"response.completed","response":{"id":"response-local","status":"completed","usage":{}}}\n\n', + { status: 200, headers: { "content-type": "text/event-stream" } }, + ); + }, + }); + const events = await Array.fromAsync( + provider.stream({ modelId: "local-responses", messages: [] }), + ); + assert.equal(events.at(-1)?.type, "completed", JSON.stringify(events)); + assert.equal(requestUrl, "http://127.0.0.1:11435/v1/responses"); + assert.equal(headers.get("authorization"), "Bearer custom-secret"); + assert.equal(headers.get("x-tenant"), "configured"); +}); + +test("refreshes dynamic catalogs only when explicitly requested and keeps providers isolated", async () => { + const calls: string[] = []; + const fetchImpl: typeof fetch = async (input) => { + const url = String(input); + calls.push(url); + if (url.endsWith("/v1/config")) { + return Response.json({ + baseUrl: "https://radius.pi.dev/v1", + models: [ + { + id: "auto", + name: "Auto", + reasoning: true, + toolUse: true, + input: ["text"], + cost: { input: 0, output: 0 }, + contextWindow: 128000, + maxTokens: 16000, + }, + ], + }); + } + const data: unknown[] = [ + { + id: "model-one", + name: "Model One", + apiDialect: "openai-chat", + architecture: { input_modalities: ["text"], output_modalities: ["text"] }, + context_length: 128000, + top_provider: { max_completion_tokens: 16000 }, + supported_parameters: ["tools"], + }, + ]; + if (url.startsWith("https://openrouter.ai/")) + data.push({ + id: "image-one", + name: "Image One", + architecture: { input_modalities: ["text", "image"], output_modalities: ["image"] }, + context_length: 128000, + top_provider: { max_completion_tokens: 16000 }, + supported_parameters: [], + }); + return Response.json({ data }, { headers: { etag: '"generation-one"' } }); + }; + const providers = [ + createOpenRouterProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + createGitHubCopilotProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + createCloudflareAiGatewayProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: fetchImpl, + }), + createRadiusProvider({ store: new InMemoryCredentialStore(), context, fetch: fetchImpl }), + ]; + const registry = new ProviderRegistry({ + catalogStore: new InMemoryCatalogStore(), + now: () => 100, + }); + for (const provider of providers) registry.register(provider); + assert.equal(calls.length, 0); + assert.equal((await registry.listModels()).models.length, 0); + assert.equal(calls.length, 0); + + const result = await registry.refresh(); + assert.deepEqual([...result.refreshedProviderIds].sort(), [ + "cloudflare-ai-gateway", + "github-copilot", + "openrouter", + "radius", + ]); + assert.equal(result.errors.size, 0); + assert.equal(calls.length, 4); + for (const provider of providers) { + const models = (await registry.listModels({ providerId: provider.id })).models; + assert.equal(models.length, 1, provider.id); + assert.equal(models[0]?.providerId, provider.id); + } + const openrouter = providers.find((provider) => provider.id === "openrouter"); + assert.equal((await openrouter?.listImageModels?.())?.length, 1); + assert.equal((await registry.listImageModels("openrouter")).length, 1); +}); + +test("dispatches a restored dynamic catalog without an implicit refresh", async () => { + const store = new InMemoryCatalogStore(); + const first = new ProviderRegistry({ catalogStore: store, now: () => 100 }); + first.register( + createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => + Response.json({ + data: [ + { + id: "restored-model", + name: "Restored Model", + architecture: { + input_modalities: ["text", "image"], + output_modalities: ["text", "image"], + }, + context_length: 128000, + top_provider: { max_completion_tokens: 16000 }, + supported_parameters: ["tools"], + }, + ], + }), + }), + ); + await first.refresh({ providerId: "openrouter" }); + + let requests = 0; + const second = new ProviderRegistry({ catalogStore: store, now: () => 200 }); + second.register( + createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => { + requests += 1; + return new Response( + [ + 'data: {"id":"response-1","model":"restored-model","choices":[{"delta":{"content":"ok"}}]}', + 'data: {"choices":[{"delta":{},"finish_reason":"stop"}]}', + "data: [DONE]", + "", + ].join("\n\n"), + { status: 200, headers: { "content-type": "text/event-stream" } }, + ); + }, + }), + ); + const restored = await second.restoreCatalogs({ providerId: "openrouter" }); + assert.deepEqual(restored.restoredProviderIds, ["openrouter"]); + assert.equal((await second.listImageModels("openrouter")).length, 1); + assert.equal(requests, 0); + await consume(second.get("openrouter"), "restored-model").catch(() => undefined); + assert.equal(requests, 0, "direct provider dispatch must not invent restored state"); + for await (const _event of second.stream("openrouter", { + modelId: "restored-model", + messages: [], + })) { + // Registry supplies the validated restored model to the provider transport. + } + assert.equal(requests, 1); +}); + +test("rejects unsafe custom endpoints and secret-shaped headers before dispatch", () => { + const source = getStaticModelCatalog("deepseek")[0]; + assert.ok(source); + for (const baseUrl of [ + "http://example.com/v1", + "https://169.254.169.254/latest", + "https://10.0.0.1/v1", + "https://user:password@example.com/v1", + "https://example.com/v1#fragment", + ]) { + assert.throws(() => + createCustomProvider({ + store: new InMemoryCredentialStore(), + context, + baseUrl, + models: [{ ...source, providerId: "custom", modelId: "unsafe" }], + }), + ); + } + for (const name of [ + "Authorization", + "Cookie", + "Proxy-Authorization", + "X-Api-Key", + "x-service-token", + ]) { + assert.throws(() => + createCustomProvider({ + store: new InMemoryCredentialStore(), + context, + baseUrl: "https://example.com/v1", + headers: { [name]: "caller-secret" }, + models: [{ ...source, providerId: "custom", modelId: "unsafe" }], + }), + ); + } +}); + +test("rejects dynamic endpoint origin changes before persistence or dispatch", async () => { + const store = new InMemoryCatalogStore(); + const requests: string[] = []; + const provider = createRadiusProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async (input) => { + requests.push(String(input)); + return Response.json({ + baseUrl: "http://169.254.169.254/latest", + models: [ + { + id: "unsafe", + name: "Unsafe", + reasoning: false, + toolUse: false, + input: ["text"], + contextWindow: 1_000, + maxTokens: 100, + }, + ], + }); + }, + }); + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register(provider); + const refreshed = await registry.refresh({ providerId: "radius" }); + assert.equal(refreshed.refreshedProviderIds.length, 0); + assert.match(refreshed.errors.get("radius")?.message ?? "", /HTTPS|origin|disallowed/); + assert.deepEqual(requests, ["https://radius.pi.dev/v1/config"]); + assert.equal(await store.read("radius"), undefined); +}); + +test("rejects incomplete dynamic rows instead of guessing compatibility", async () => { + const factories = [ + () => + createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => Response.json({ data: [{ id: "x", name: "X" }] }), + }), + () => + createGitHubCopilotProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => Response.json({ data: [{ id: "x", name: "X" }] }), + }), + () => + createCloudflareAiGatewayProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => Response.json({ data: [{ id: "x", name: "X" }] }), + }), + () => + createRadiusProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => + Response.json({ baseUrl: "https://radius.pi.dev/v1", models: [{ id: "x", name: "X" }] }), + }), + ]; + for (const factory of factories) { + const provider = factory(); + const registry = new ProviderRegistry(); + registry.register(provider); + const result = await registry.refresh({ providerId: provider.id }); + assert.equal(result.refreshedProviderIds.length, 0, provider.id); + assert.ok(result.errors.has(provider.id), provider.id); + } +}); + +test("rejects dynamic model counts above the publication limit", async () => { + const row = { + id: "model", + name: "Model", + architecture: { input_modalities: ["text"], output_modalities: ["text"] }, + context_length: 1_000, + top_provider: { max_completion_tokens: 100 }, + supported_parameters: [], + }; + const provider = createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => Response.json({ data: Array.from({ length: 10_001 }, () => row) }), + }); + const registry = new ProviderRegistry(); + registry.register(provider); + const result = await registry.refresh({ providerId: "openrouter" }); + assert.equal(result.refreshedProviderIds.length, 0); + assert.match(result.errors.get("openrouter")?.message ?? "", /bounded model array/); +}); + +test("revalidates restored dynamic origins before credentialed dispatch", async () => { + const source = getStaticModelCatalog("deepseek")[0]; + assert.ok(source); + const store = new InMemoryCatalogStore(); + await store.write("openrouter", { + version: 1, + providerId: "openrouter", + generation: 1, + checkedAt: 1, + updatedAt: 1, + source: { id: "openrouter-models", kind: "provider_api" }, + models: [ + { + ...source, + providerId: "openrouter", + modelId: "restored-foreign-origin", + endpoint: { type: "fixed", baseUrl: "https://attacker.example/v1" }, + }, + ], + }); + let fetches = 0; + const provider = createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async () => { + fetches += 1; + throw new Error("must not dispatch"); + }, + }); + const registry = new ProviderRegistry({ catalogStore: store }); + registry.register(provider); + await registry.restoreCatalogs({ providerId: "openrouter" }); + const events = await Array.fromAsync( + registry.stream("openrouter", { modelId: "restored-foreign-origin", messages: [] }), + ); + assert.equal(events.at(-1)?.type, "error"); + assert.equal(fetches, 0); +}); + +test("uses distinct Anthropic environment authentication headers", async () => { + for (const [name, expected] of [ + ["ANTHROPIC_API_KEY", { apiKey: "fixture", authorization: null }], + ["ANTHROPIC_OAUTH_TOKEN", { apiKey: null, authorization: "Bearer fixture" }], + ] as const) { + let headers = new Headers(); + const provider = createAnthropicProvider({ + store: new InMemoryCredentialStore(), + context: { + env: (key) => (key === name ? "fixture" : undefined), + fileExists: () => Promise.resolve(false), + }, + fetch: async (_input, init) => { + headers = new Headers(init?.headers); + return new Response('data: {"type":"message_stop"}\n\n', { status: 200 }); + }, + }); + const model = (await provider.listModels())[0]; + assert.ok(model); + await consume(provider, model.modelId); + assert.equal(headers.get("x-api-key"), expected.apiKey); + assert.equal(headers.get("authorization"), expected.authorization); + if (name === "ANTHROPIC_OAUTH_TOKEN") + assert.match(headers.get("anthropic-beta") ?? "", /oauth/); + } +}); + +test("times out non-cancellable image transport without retrying", async () => { + let imageFetches = 0; + const provider = createOpenRouterProvider({ + store: new InMemoryCredentialStore(), + context, + fetch: async (input) => { + if (String(input).endsWith("/models")) { + return Response.json({ + data: [ + { + id: "image-timeout", + name: "Image timeout", + architecture: { input_modalities: ["text"], output_modalities: ["image"] }, + context_length: 1_000, + top_provider: { max_completion_tokens: 100 }, + supported_parameters: [], + }, + ], + }); + } + imageFetches += 1; + return new Promise(() => undefined); + }, + }); + const registry = new ProviderRegistry(); + registry.register(provider); + await registry.refresh({ providerId: "openrouter" }); + assert.ok(provider.generateImages); + const guard = setTimeout(() => undefined, 100); + try { + await assert.rejects( + provider.generateImages({ + modelId: "image-timeout", + prompt: "fixture", + timeoutMs: 1, + maxRetries: 0, + writeBlob: async () => { + throw new Error("not reached"); + }, + }), + { name: "TimeoutError" }, + ); + } finally { + clearTimeout(guard); + } + assert.equal(imageFetches, 1); +}); diff --git a/packages/ai/test/request-configuration.test.ts b/packages/ai/test/request-configuration.test.ts index 5b88644..43e1676 100644 --- a/packages/ai/test/request-configuration.test.ts +++ b/packages/ai/test/request-configuration.test.ts @@ -63,7 +63,7 @@ test("the port records fitted configuration before dispatch and preserves reason }); const port = modelPortForSession(provider, { modelId: model.modelId, - thinkingLevel: "max", + thinkingLevel: "high", requestSettings: { maxOutputTokens: null, httpIdleTimeoutMs: 0 }, }); let recorded = false; @@ -81,6 +81,6 @@ test("the port records fitted configuration before dispatch and preserves reason } assert.equal(recorded, true); assert.equal(provider.requests[0]?.maxOutputTokens, 5904); - assert.equal(provider.requests[0]?.thinkingLevel, "max"); + assert.equal(provider.requests[0]?.thinkingLevel, "high"); assert.equal(provider.requests[0]?.httpIdleTimeoutMs, 0); }); diff --git a/packages/ai/test/request-preparation.test.ts b/packages/ai/test/request-preparation.test.ts new file mode 100644 index 0000000..3b55da4 --- /dev/null +++ b/packages/ai/test/request-preparation.test.ts @@ -0,0 +1,491 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createHash } from "node:crypto"; +import test from "node:test"; + +import { + makeFakeModelInfo, + prepareModelRequest, + RequestPreparationError, + validateModelCatalog, + type ModelInfo, + type ModelRequest, +} from "../src/index.ts"; + +const textRequest = (overrides: Partial = {}): ModelRequest => ({ + modelId: "target-model", + messages: [{ role: "user", content: [{ type: "text", text: "hello" }] }], + ...overrides, +}); + +function targetModel(overrides: Partial = {}): ModelInfo { + return makeFakeModelInfo({ + providerId: "target-provider", + modelId: "target-model", + apiDialect: "openai-chat", + compatibility: { + dialect: "openai-chat", + supportsStrictTools: true, + supportsGrammarTools: true, + }, + ...overrides, + }); +} + +test("normalizes history tool identifiers and preserves canonical tool identity", async () => { + const foreign = { + providerId: "source-provider", + apiDialect: "openai-responses", + modelId: "source-model", + } as const; + const request = textRequest({ + tools: [ + { + name: "browser.open", + description: "Open a page", + inputSchema: { type: "object" }, + }, + ], + messages: [ + { role: "user", content: [{ type: "text", text: "open it" }] }, + { + role: "assistant", + content: [ + { + type: "thinking", + text: "I should use the browser", + signature: { ...foreign, value: "opaque-thinking" }, + }, + { + type: "text", + text: "Opening", + continuation: { ...foreign, itemId: "message-item" }, + }, + ], + toolCalls: [ + { + callId: "call|with/invalid+characters".repeat(4), + name: "browser.open", + input: { url: "https://example.test" }, + signature: { ...foreign, value: "opaque-tool" }, + }, + ], + }, + { + role: "tool", + callId: "call|with/invalid+characters".repeat(4), + name: "browser.open", + content: [{ type: "text", text: "done" }], + isError: false, + }, + ], + }); + + const prepared = await prepareModelRequest(targetModel(), request); + const tool = prepared.tools?.[0]; + const assistant = prepared.messages[1]; + const result = prepared.messages[2]; + + assert.equal(tool?.canonicalName, "browser.open"); + assert.equal(tool?.name, "browser_open"); + assert.equal(assistant?.role, "assistant"); + assert.equal(result?.role, "tool"); + if (assistant?.role !== "assistant" || result?.role !== "tool") assert.fail("unexpected history"); + const call = assistant.toolCalls?.[0]; + assert.match(call?.callId ?? "", /^[A-Za-z0-9_-]{1,64}$/); + assert.equal(call?.canonicalName, "browser.open"); + assert.equal( + call?.canonicalCallId, + request.messages[1]?.role === "assistant" + ? request.messages[1].toolCalls?.[0]?.callId + : undefined, + ); + assert.equal(result.callId, call?.callId); + assert.equal(result.canonicalCallId, call?.canonicalCallId); + assert.equal(result.name, "browser_open"); + assert.deepEqual(prepared.preparation.sanitizations, [ + { + path: "request.messages[1].content[0].signature", + reason: "foreign-provider-signature", + }, + { + path: "request.messages[1].content[1].continuation", + reason: "foreign-provider-continuation", + }, + { + path: "request.messages[1].toolCalls[0].signature", + reason: "foreign-provider-signature", + }, + ]); + const thinking = assistant.content[0]; + const text = assistant.content[1]; + assert.ok(thinking); + assert.ok(text); + assert.equal("signature" in thinking, false); + assert.equal("continuation" in text, false); +}); + +test("retains replay metadata only for its exact issuing model", async () => { + const identity = { + providerId: "target-provider", + apiDialect: "openai-chat", + modelId: "target-model", + } as const; + const prepared = await prepareModelRequest( + targetModel(), + textRequest({ + messages: [ + { + role: "assistant", + content: [ + { type: "thinking", text: "", signature: { ...identity, value: "signed" } }, + { + type: "text", + text: "answer", + continuation: { ...identity, responseId: "response-1", itemId: "item-1" }, + }, + ], + origin: identity, + continuation: { ...identity, responseId: "response-1" }, + }, + ], + }), + ); + + const message = prepared.messages[0]; + assert.equal(message?.role, "assistant"); + if (message?.role !== "assistant") assert.fail("unexpected history"); + assert.equal( + message.content[0]?.type === "thinking" ? message.content[0].signature?.value : undefined, + "signed", + ); + assert.equal(message.continuation?.responseId, "response-1"); + assert.deepEqual(prepared.preparation.sanitizations, []); +}); + +test("loads each image blob once and verifies size and digest", async () => { + const bytes = new Uint8Array([1, 2, 3, 4]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + let reads = 0; + const reference = { sha256, mediaType: "image/png", sizeBytes: bytes.byteLength }; + const prepared = await prepareModelRequest( + targetModel({ capabilities: { toolUse: true, structuredOutput: true, imageInput: true } }), + textRequest({ + messages: [ + { + role: "user", + content: [ + { type: "blob", blob: reference }, + { type: "blob", blob: reference }, + ], + }, + ], + readBlob: async () => { + reads += 1; + return bytes; + }, + }), + ); + + assert.equal(reads, 1); + assert.equal(prepared.preparation.blobs.size, 1); + assert.deepEqual(await prepared.readBlob?.(reference), bytes); +}); + +test("rejects unsupported media and changed blob bytes", async () => { + const bytes = new Uint8Array([1]); + const sha256 = createHash("sha256").update(bytes).digest("hex"); + const request = textRequest({ + messages: [ + { + role: "user", + content: [{ type: "blob", blob: { sha256, mediaType: "image/png", sizeBytes: 1 } }], + }, + ], + readBlob: async () => new Uint8Array([2]), + }); + await assert.rejects( + () => prepareModelRequest(targetModel(), request), + /does not support: imageInput/, + ); + await assert.rejects( + () => + prepareModelRequest( + targetModel({ capabilities: { toolUse: true, structuredOutput: true, imageInput: true } }), + request, + ), + /content hash does not match/, + ); +}); + +test("prepares strict and grammar constrained tools without changing canonical names", async () => { + const prepared = await prepareModelRequest( + targetModel(), + textRequest({ + tools: [ + { + name: "search.docs", + description: "Search documentation", + inputSchema: { + type: "object", + properties: { query: { type: "string" }, limit: { type: "number" } }, + required: ["query"], + }, + constraint: { type: "json-schema", strict: "require" }, + }, + { + name: "write.expression", + description: "Write an expression", + inputSchema: { + type: "object", + properties: { expression: { type: "string" } }, + required: ["expression"], + }, + constraint: { type: "grammar", variants: { lark: "start: NUMBER" } }, + }, + ], + }), + ); + + const strict = prepared.tools?.[0]; + const grammar = prepared.tools?.[1]; + assert.equal(strict?.canonicalName, "search.docs"); + assert.equal(strict?.name, "search_docs"); + assert.deepEqual(strict?.preparedConstraint, { type: "json-schema", strict: true }); + assert.deepEqual(strict?.inputSchema.required, ["query", "limit"]); + assert.equal(strict?.inputSchema.additionalProperties, false); + assert.deepEqual(grammar?.preparedConstraint, { + type: "grammar", + format: "lark", + definition: "start: NUMBER", + inputProperty: "expression", + }); +}); + +test("rejects required constrained sampling that the model cannot honor", async () => { + const unsupported = targetModel({ + compatibility: { + dialect: "openai-chat", + supportsStrictTools: false, + supportsGrammarTools: false, + }, + }); + const strictRequest = textRequest({ + tools: [ + { + name: "strict", + description: "Strict", + inputSchema: { type: "object" }, + constraint: { type: "json-schema", strict: "require" }, + }, + ], + }); + await assert.rejects( + () => prepareModelRequest(unsupported, strictRequest), + /strict schemas unsupported/, + ); + await assert.rejects( + () => + prepareModelRequest( + unsupported, + textRequest({ + tools: [ + { + name: "grammar", + description: "Grammar", + inputSchema: { + type: "object", + properties: { value: { type: "string" } }, + required: ["value"], + }, + constraint: { type: "grammar", variants: { regex: ".+" } }, + }, + ], + }), + ), + /grammar tools unsupported/, + ); +}); + +test("resolves reasoning, cache placement, and validated sampling controls", async () => { + const model = targetModel({ + maxOutputTokens: 20_000, + thinkingLevelMap: { low: null, medium: "standard", high: "high" }, + cache: { + supported: true, + defaultRetention: "short", + supportedRetentions: ["none", "short", "long"], + }, + sampling: { + supported: [ + "temperature", + "topP", + "topK", + "minP", + "frequencyPenalty", + "presencePenalty", + "repetitionPenalty", + "seed", + ], + customFields: ["mirostat"], + }, + compatibility: { + dialect: "openai-chat", + supportsStrictTools: true, + supportsGrammarTools: true, + thinkingTokenBudgetField: "thinking_budget_tokens", + cacheControlFormat: "anthropic", + supportsLongCacheRetention: true, + }, + }); + const prepared = await prepareModelRequest( + model, + textRequest({ + system: "stable system", + thinkingLevel: "low", + thinkingBudgets: { medium: 2_000 }, + maxOutputTokens: 3_000, + cache: { retention: "long", sessionId: "session-1" }, + sampling: { + temperature: 0.2, + topP: 0.9, + topK: 20, + minP: 0.1, + frequencyPenalty: -0.5, + presencePenalty: 0.5, + repetitionPenalty: 1.1, + seed: 7, + custom: { mirostat: 2 }, + }, + tools: [{ name: "read", description: "Read", inputSchema: { type: "object" } }], + }), + ); + + assert.deepEqual(prepared.preparation.reasoning, { + requested: "low", + effective: "medium", + clamped: true, + tokenBudget: 2_000, + }); + assert.equal(prepared.thinkingLevel, "medium"); + assert.equal(prepared.maxOutputTokens, 5_000); + assert.deepEqual(prepared.preparation.cache, { + retention: "long", + sessionId: "session-1", + placements: [ + { target: "system" }, + { target: "tool", toolIndex: 0 }, + { target: "message-content", messageIndex: 0, contentIndex: 0 }, + ], + }); + assert.deepEqual(prepared.sampling, { + temperature: 0.2, + topP: 0.9, + topK: 20, + minP: 0.1, + frequencyPenalty: -0.5, + presencePenalty: 0.5, + repetitionPenalty: 1.1, + seed: 7, + custom: { mirostat: 2 }, + }); +}); + +test("rejects invalid history, sampling, cache, and authorization metadata loudly", async () => { + const model = targetModel(); + await assert.rejects( + () => + prepareModelRequest( + model, + textRequest({ + messages: [ + { + role: "tool", + callId: "missing", + name: "read", + content: [{ type: "text", text: "result" }], + isError: false, + }, + ], + }), + ), + /does not match a preceding unresolved tool call/, + ); + await assert.rejects( + () => prepareModelRequest(model, textRequest({ sampling: { topP: 2 } })), + /sampling.topP must be between 0 and 1/, + ); + await assert.rejects( + () => prepareModelRequest(model, textRequest({ sampling: { topK: 20 } })), + /sampling.topK is unsupported by this model/, + ); + await assert.rejects( + () => prepareModelRequest(model, textRequest({ sampling: { custom: { mirostat: 2 } } })), + /sampling.custom.mirostat is unsupported by this model/, + ); + await assert.rejects( + () => + prepareModelRequest( + model, + textRequest({ cache: { retention: "none", sessionId: "ignored" } }), + ), + /requires prompt caching to be enabled/, + ); + await assert.rejects( + () => prepareModelRequest(model, textRequest({ metadata: { accessToken: "must-not-pass" } })), + (error) => + error instanceof RequestPreparationError && + error.path === "request.metadata.accessToken" && + !JSON.stringify(error).includes("must-not-pass"), + ); +}); + +test("validates model sampling declarations in catalog metadata", () => { + assert.throws( + () => + validateModelCatalog([ + targetModel({ + sampling: { + supported: ["temperature", "temperature"], + customFields: ["accessToken"], + }, + }), + ]), + (error) => + error instanceof Error && + error.message.includes("invalid sampling support") && + error.message.includes("invalid custom sampling fields"), + ); +}); + +test("rejects foreign redacted reasoning instead of dropping opaque content", async () => { + await assert.rejects( + () => + prepareModelRequest( + targetModel(), + textRequest({ + messages: [ + { + role: "assistant", + content: [ + { + type: "thinking", + text: "", + redacted: true, + signature: { + providerId: "other", + apiDialect: "anthropic-messages", + modelId: "other-model", + value: "opaque", + }, + }, + ], + }, + ], + }), + ), + /redacted reasoning that cannot be replayed/, + ); +}); diff --git a/packages/ai/test/request-timeout.test.ts b/packages/ai/test/request-timeout.test.ts index 911dbde..eeb0f8d 100644 --- a/packages/ai/test/request-timeout.test.ts +++ b/packages/ai/test/request-timeout.test.ts @@ -23,7 +23,12 @@ async function provider(t: TestContext, handle: (response: ServerResponse) => vo id: "timeout-test", displayName: "Timeout test", authMethods: ["keyless"], - models: [makeFakeModelInfo()], + models: [ + makeFakeModelInfo({ + apiDialect: "openai-responses", + compatibility: { dialect: "openai-responses" }, + }), + ], resolveAuth: async () => ({ auth: {}, source: "test", secretValues: [] }), endpoint: { url: () => `http://127.0.0.1:${address.port}/responses`, diff --git a/packages/ai/test/sse.test.ts b/packages/ai/test/sse.test.ts index 023c10d..0787dde 100644 --- a/packages/ai/test/sse.test.ts +++ b/packages/ai/test/sse.test.ts @@ -4,7 +4,15 @@ import assert from "node:assert/strict"; import test from "node:test"; -import { decodeSseStream, type SseFrame } from "../src/index.ts"; +import { + decodeSseStream, + MAX_SSE_DATA_LINES, + MAX_SSE_EVENT_NAME_BYTES, + MAX_SSE_FRAME_DATA_BYTES, + MAX_SSE_LINE_BYTES, + MAX_SSE_TOTAL_BYTES, + type SseFrame, +} from "../src/index.ts"; async function* chunks(parts: readonly string[]): AsyncGenerator { for (const part of parts) yield new TextEncoder().encode(part); @@ -44,3 +52,29 @@ test("flushes a final frame that was never newline-terminated", async () => { test("emits nothing for empty input", async () => { assert.deepEqual(await collect([]), []); }); + +test("rejects oversized pending lines and total responses across chunk boundaries", async () => { + await assert.rejects( + collect(["data: ", "x".repeat(MAX_SSE_LINE_BYTES)]), + /pending line|line exceeds/, + ); + const comment = `:${"x".repeat(MAX_SSE_LINE_BYTES - 2)}\n`; + const chunks = Array.from( + { length: Math.ceil(MAX_SSE_TOTAL_BYTES / Buffer.byteLength(comment)) + 1 }, + () => comment, + ); + await assert.rejects(collect(chunks), /response exceeds/); +}); + +test("rejects oversized event names, data-line counts, and joined frame data", async () => { + await assert.rejects( + collect([`event: ${"x".repeat(MAX_SSE_EVENT_NAME_BYTES + 1)}\n\n`]), + /event name exceeds/, + ); + await assert.rejects( + collect([`${Array.from({ length: MAX_SSE_DATA_LINES + 1 }, () => "data: x").join("\n")}\n\n`]), + /data-line limit/, + ); + const line = `data: ${"x".repeat(Math.floor(MAX_SSE_FRAME_DATA_BYTES / 5) + 1)}\n`; + await assert.rejects(collect([line, line, line, line, line, "\n"]), /frame data exceeds/); +}); diff --git a/packages/ai/test/static-openai-chat-providers.test.ts b/packages/ai/test/static-openai-chat-providers.test.ts new file mode 100644 index 0000000..5617b88 --- /dev/null +++ b/packages/ai/test/static-openai-chat-providers.test.ts @@ -0,0 +1,477 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + ANT_LING_BASE_URL, + BASETEN_BASE_URL, + CEREBRAS_BASE_URL, + collectModelStream, + createAntLingProvider, + createBasetenProvider, + createCerebrasProvider, + createFireworksProvider, + createGroqProvider, + createHuggingFaceProvider, + createMiniMaxCnProvider, + createMiniMaxProvider, + createMoonshotAiCnProvider, + createMoonshotAiProvider, + createNvidiaProvider, + createQwenTokenPlanCnProvider, + createQwenTokenPlanIndividualProvider, + createQwenTokenPlanProvider, + createStaticOpenAiChatProvider, + createTogetherProvider, + createVercelAiGatewayProvider, + createXaiProvider, + createXiaomiProvider, + createXiaomiTokenPlanAmsProvider, + createXiaomiTokenPlanCnProvider, + createXiaomiTokenPlanSgpProvider, + createZaiCodingCnProvider, + createZaiProvider, + FIREWORKS_BASE_URL, + GROQ_BASE_URL, + getStaticModelCatalog, + HUGGINGFACE_BASE_URL, + InMemoryCredentialStore, + listBuiltinCatalogProviders, + MINIMAX_BASE_URL, + MINIMAX_CN_BASE_URL, + MOONSHOTAI_BASE_URL, + MOONSHOTAI_CN_BASE_URL, + NVIDIA_BASE_URL, + ProviderRegistry, + QWEN_TOKEN_PLAN_BASE_URL, + QWEN_TOKEN_PLAN_CN_BASE_URL, + QWEN_TOKEN_PLAN_INDIVIDUAL_BASE_URL, + type StaticOpenAiChatProviderDefinition, + type StaticOpenAiChatProviderOptions, + TOGETHER_BASE_URL, + VERCEL_AI_GATEWAY_BASE_URL, + XAI_BASE_URL, + XIAOMI_BASE_URL, + XIAOMI_TOKEN_PLAN_AMS_BASE_URL, + XIAOMI_TOKEN_PLAN_CN_BASE_URL, + XIAOMI_TOKEN_PLAN_SGP_BASE_URL, + ZAI_BASE_URL, + ZAI_CODING_CN_BASE_URL, +} from "../src/index.ts"; + +interface ProviderCase { + readonly id: string; + readonly displayName: string; + readonly environmentVariable: string; + readonly baseUrl: string; + readonly expectedModels: number; + readonly region?: string; + readonly regionFamily?: string; + readonly create: ( + options: StaticOpenAiChatProviderOptions, + ) => ReturnType; +} + +const PROVIDERS: readonly ProviderCase[] = [ + { + id: "groq", + displayName: "Groq", + environmentVariable: "GROQ_API_KEY", + baseUrl: GROQ_BASE_URL, + expectedModels: 7, + create: createGroqProvider, + }, + { + id: "cerebras", + displayName: "Cerebras", + environmentVariable: "CEREBRAS_API_KEY", + baseUrl: CEREBRAS_BASE_URL, + expectedModels: 2, + create: createCerebrasProvider, + }, + { + id: "nvidia", + displayName: "NVIDIA NIM", + environmentVariable: "NVIDIA_API_KEY", + baseUrl: NVIDIA_BASE_URL, + expectedModels: 64, + create: createNvidiaProvider, + }, + { + id: "baseten", + displayName: "Baseten", + environmentVariable: "BASETEN_API_KEY", + baseUrl: BASETEN_BASE_URL, + expectedModels: 22, + create: createBasetenProvider, + }, + { + id: "huggingface", + displayName: "Hugging Face", + environmentVariable: "HF_TOKEN", + baseUrl: HUGGINGFACE_BASE_URL, + expectedModels: 70, + create: createHuggingFaceProvider, + }, + { + id: "zai", + displayName: "Z.AI", + environmentVariable: "ZAI_API_KEY", + baseUrl: ZAI_BASE_URL, + expectedModels: 16, + region: "global", + regionFamily: "zai", + create: createZaiProvider, + }, + { + id: "zai-coding-cn", + displayName: "Z.AI Coding China", + environmentVariable: "ZAI_CODING_CN_API_KEY", + baseUrl: ZAI_CODING_CN_BASE_URL, + expectedModels: 10, + region: "cn", + regionFamily: "zai", + create: createZaiCodingCnProvider, + }, + { + id: "minimax", + displayName: "MiniMax", + environmentVariable: "MINIMAX_API_KEY", + baseUrl: MINIMAX_BASE_URL, + expectedModels: 7, + region: "global", + regionFamily: "minimax", + create: createMiniMaxProvider, + }, + { + id: "minimax-cn", + displayName: "MiniMax China", + environmentVariable: "MINIMAX_CN_API_KEY", + baseUrl: MINIMAX_CN_BASE_URL, + expectedModels: 7, + region: "cn", + regionFamily: "minimax", + create: createMiniMaxCnProvider, + }, + { + id: "moonshotai", + displayName: "Moonshot AI", + environmentVariable: "MOONSHOT_API_KEY", + baseUrl: MOONSHOTAI_BASE_URL, + expectedModels: 10, + region: "global", + regionFamily: "moonshotai", + create: createMoonshotAiProvider, + }, + { + id: "moonshotai-cn", + displayName: "Moonshot AI China", + environmentVariable: "MOONSHOT_API_KEY", + baseUrl: MOONSHOTAI_CN_BASE_URL, + expectedModels: 10, + region: "cn", + regionFamily: "moonshotai", + create: createMoonshotAiCnProvider, + }, + { + id: "qwen-token-plan", + displayName: "Qwen Token Plan", + environmentVariable: "QWEN_TOKEN_PLAN_API_KEY", + baseUrl: QWEN_TOKEN_PLAN_BASE_URL, + expectedModels: 19, + region: "sgp", + regionFamily: "qwen-token-plan", + create: createQwenTokenPlanProvider, + }, + { + id: "qwen-token-plan-cn", + displayName: "Qwen Token Plan China", + environmentVariable: "QWEN_TOKEN_PLAN_CN_API_KEY", + baseUrl: QWEN_TOKEN_PLAN_CN_BASE_URL, + expectedModels: 19, + region: "cn", + regionFamily: "qwen-token-plan", + create: createQwenTokenPlanCnProvider, + }, + { + id: "vercel-ai-gateway", + displayName: "Vercel AI Gateway", + environmentVariable: "AI_GATEWAY_API_KEY", + baseUrl: VERCEL_AI_GATEWAY_BASE_URL, + expectedModels: 229, + create: createVercelAiGatewayProvider, + }, + { + id: "fireworks", + displayName: "Fireworks AI", + environmentVariable: "FIREWORKS_API_KEY", + baseUrl: FIREWORKS_BASE_URL, + expectedModels: 20, + create: createFireworksProvider, + }, + { + id: "together", + displayName: "Together AI", + environmentVariable: "TOGETHER_API_KEY", + baseUrl: TOGETHER_BASE_URL, + expectedModels: 32, + create: createTogetherProvider, + }, + { + id: "qwen-token-plan-individual", + displayName: "Qwen Token Plan Individual", + environmentVariable: "QWEN_TOKEN_PLAN_API_KEY", + baseUrl: QWEN_TOKEN_PLAN_INDIVIDUAL_BASE_URL, + expectedModels: 19, + create: createQwenTokenPlanIndividualProvider, + }, + { + id: "xiaomi", + displayName: "Xiaomi MiMo", + environmentVariable: "XIAOMI_API_KEY", + baseUrl: XIAOMI_BASE_URL, + expectedModels: 6, + region: "global", + regionFamily: "xiaomi", + create: createXiaomiProvider, + }, + { + id: "xiaomi-token-plan-cn", + displayName: "Xiaomi Token Plan China", + environmentVariable: "XIAOMI_TOKEN_PLAN_CN_API_KEY", + baseUrl: XIAOMI_TOKEN_PLAN_CN_BASE_URL, + expectedModels: 3, + region: "cn", + regionFamily: "xiaomi", + create: createXiaomiTokenPlanCnProvider, + }, + { + id: "xiaomi-token-plan-ams", + displayName: "Xiaomi Token Plan Amsterdam", + environmentVariable: "XIAOMI_TOKEN_PLAN_AMS_API_KEY", + baseUrl: XIAOMI_TOKEN_PLAN_AMS_BASE_URL, + expectedModels: 3, + region: "ams", + regionFamily: "xiaomi", + create: createXiaomiTokenPlanAmsProvider, + }, + { + id: "xiaomi-token-plan-sgp", + displayName: "Xiaomi Token Plan Singapore", + environmentVariable: "XIAOMI_TOKEN_PLAN_SGP_API_KEY", + baseUrl: XIAOMI_TOKEN_PLAN_SGP_BASE_URL, + expectedModels: 3, + region: "sgp", + regionFamily: "xiaomi", + create: createXiaomiTokenPlanSgpProvider, + }, + { + id: "ant-ling", + displayName: "Ant Ling", + environmentVariable: "ANT_LING_API_KEY", + baseUrl: ANT_LING_BASE_URL, + expectedModels: 4, + create: createAntLingProvider, + }, + { + id: "xai", + displayName: "xAI", + environmentVariable: "XAI_API_KEY", + baseUrl: XAI_BASE_URL, + expectedModels: 6, + create: createXaiProvider, + }, +]; + +function context(environment: Readonly> = {}) { + return { + env: (name: string) => environment[name], + fileExists: () => Promise.resolve(false), + }; +} + +function streamResponse(modelId: string): Response { + const body = [ + { id: "response-1", model: modelId, choices: [{ delta: { content: "ok" } }] }, + { choices: [{ delta: {}, finish_reason: "stop" }] }, + "[DONE]", + ] + .map((value) => `data: ${typeof value === "string" ? value : JSON.stringify(value)}\n\n`) + .join(""); + return new Response(body, { + status: 200, + headers: { "content-type": "text/event-stream" }, + }); +} + +test("constructs and lists static Chat providers without credential or network work", async () => { + for (const providerCase of PROVIDERS) { + let environmentReads = 0; + let fetches = 0; + const provider = providerCase.create({ + store: new InMemoryCredentialStore(), + context: { + env: () => { + environmentReads += 1; + return undefined; + }, + fileExists: () => Promise.resolve(false), + }, + fetch: async () => { + fetches += 1; + throw new Error("unexpected network request"); + }, + }); + + const models = await provider.listModels(); + assert.equal(provider.id, providerCase.id); + assert.equal(provider.displayName, providerCase.displayName); + assert.deepEqual( + provider.authMethods, + providerCase.id === "xai" ? ["environment", "file", "oauth"] : ["environment", "file"], + ); + const catalogProvider = listBuiltinCatalogProviders().find( + (candidate) => candidate.id === providerCase.id, + ); + assert.equal(catalogProvider?.catalogKind, "static"); + assert.equal(catalogProvider?.region, providerCase.region); + assert.equal(catalogProvider?.regionFamily, providerCase.regionFamily); + assert.equal("refreshModels" in provider, false); + assert.equal(models.length, providerCase.expectedModels); + assert.equal( + models.every((model) => model.providerId === providerCase.id), + true, + ); + assert.equal( + models.every((model) => model.apiDialect === "openai-chat"), + true, + ); + assert.equal( + models.every( + (model) => + model.endpoint?.type === "fixed" && model.endpoint.baseUrl === providerCase.baseUrl, + ), + true, + ); + assert.equal(environmentReads, 0); + assert.equal(fetches, 0); + } +}); + +test("resolves each provider environment key through provider-owned authentication", async () => { + for (const providerCase of PROVIDERS) { + const provider = providerCase.create({ + store: new InMemoryCredentialStore(), + context: context({ [providerCase.environmentVariable]: `${providerCase.id}-secret` }), + }); + const authentication = provider.authentication; + if (authentication === undefined) throw new Error(`${providerCase.id} authentication missing`); + + assert.deepEqual(await authentication.resolve(), { + auth: { apiKey: `${providerCase.id}-secret` }, + source: providerCase.environmentVariable, + secretValues: [`${providerCase.id}-secret`], + }); + } +}); + +test("registers and dispatches every provider through the shared Chat transport", async () => { + for (const providerCase of PROVIDERS) { + let requestUrl = ""; + let requestInit: RequestInit | undefined; + const models = getStaticModelCatalog(providerCase.id); + const model = models[0]; + if (model === undefined) throw new Error(`${providerCase.id} catalog is empty`); + const provider = providerCase.create({ + store: new InMemoryCredentialStore(), + context: context({ [providerCase.environmentVariable]: `${providerCase.id}-secret` }), + fetch: async (input, init) => { + requestUrl = String(input); + requestInit = init; + return streamResponse(model.modelId); + }, + now: () => 100, + }); + const registry = new ProviderRegistry(); + registry.register(provider); + + const result = await collectModelStream( + registry.stream(providerCase.id, { modelId: model.modelId, messages: [] }), + ); + + assert.equal(requestUrl, `${providerCase.baseUrl}/chat/completions`); + assert.equal( + new Headers(requestInit?.headers).get("authorization"), + `Bearer ${providerCase.id}-secret`, + ); + assert.deepEqual(JSON.parse(String(requestInit?.body)), { + model: model.modelId, + messages: [], + stream: true, + stream_options: { include_usage: true }, + }); + assert.deepEqual(result.events[0], { type: "text_delta", text: "ok", contentIndex: 0 }); + assert.equal(result.terminal.type, "completed"); + assert.equal(result.terminal.response?.providerId, providerCase.id); + assert.equal(result.terminal.response?.requestedModelId, model.modelId); + } +}); + +test("rejects empty, foreign, wrong-dialect, unsafe-header, and mismatched catalogs", () => { + const definition: StaticOpenAiChatProviderDefinition = { + id: "strict-provider", + displayName: "Strict Provider", + apiKeyDisplayName: "Strict Provider API key", + environmentVariables: ["STRICT_PROVIDER_API_KEY"], + baseUrl: "https://strict.example/v1", + }; + const source = getStaticModelCatalog("groq")[0]; + if (source === undefined) throw new Error("Groq catalog is empty"); + const valid = { + ...source, + providerId: definition.id, + endpoint: { type: "fixed", baseUrl: definition.baseUrl } as const, + }; + const options = { + store: new InMemoryCredentialStore(), + context: context(), + }; + + assert.throws( + () => createStaticOpenAiChatProvider(definition, { ...options, models: [] }), + /no static models/, + ); + assert.throws( + () => createStaticOpenAiChatProvider(definition, { ...options, models: [source] }), + /owned by groq/, + ); + assert.throws( + () => + createStaticOpenAiChatProvider(definition, { + ...options, + models: [{ ...valid, apiDialect: "anthropic-messages" }], + }), + /does not use openai-chat/, + ); + assert.throws( + () => + createStaticOpenAiChatProvider(definition, { + ...options, + models: [{ ...valid, endpoint: { type: "fixed", baseUrl: "https://other.example/v1" } }], + }), + /unexpected endpoint/, + ); + assert.throws( + () => + createStaticOpenAiChatProvider(definition, { + ...options, + models: [{ ...valid, headers: { authorization: "unsafe" } }], + }), + /unsafe static header/, + ); + assert.doesNotThrow(() => + createStaticOpenAiChatProvider(definition, { ...options, models: [valid] }), + ); +}); diff --git a/packages/ai/test/stream.test.ts b/packages/ai/test/stream.test.ts index 55c0f7b..cbffcde 100644 --- a/packages/ai/test/stream.test.ts +++ b/packages/ai/test/stream.test.ts @@ -1,4 +1,5 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 import assert from "node:assert/strict"; @@ -13,6 +14,15 @@ test("passes a well-formed stream through and stops at the terminal event", asyn let pulledPastTerminal = false; async function* source(): AsyncGenerator { yield { type: "thinking_delta", text: "hmm" }; + yield { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + signature: "opaque", + }; yield { type: "text_delta", text: "hello" }; yield { type: "tool_call", callId: "call-1", name: "shell", input: { command: "true" } }; yield completed; @@ -21,7 +31,7 @@ test("passes a well-formed stream through and stops at the terminal event", asyn } const { events, terminal } = await collectModelStream(source()); - assert.equal(events.length, 4); + assert.equal(events.length, 5); assert.deepEqual(terminal, completed); assert.equal(pulledPastTerminal, false); }); @@ -41,6 +51,7 @@ test("converts a thrown provider error into an error terminal", async () => { retryable: false, category: "stream_interrupted", requestPhase: "streaming", + partial: true, }); }); @@ -54,6 +65,7 @@ test("converts a silently ended stream into an error terminal", async () => { if (terminal.type === "error") { assert.equal(terminal.code, "provider_stream_truncated"); assert.equal(terminal.retryable, false); + assert.equal(terminal.partial, true); } }); @@ -66,7 +78,7 @@ test("reports an aborted terminal when the request signal fired", async () => { } const { terminal } = await collectModelStream(source(), controller.signal); - assert.deepEqual(terminal, { type: "aborted" }); + assert.deepEqual(terminal, { type: "aborted", partial: true }); }); test("normalizeModelStream yields nothing after a terminal event", async () => { diff --git a/packages/ai/test/subscription-auth.test.ts b/packages/ai/test/subscription-auth.test.ts new file mode 100644 index 0000000..7a6f588 --- /dev/null +++ b/packages/ai/test/subscription-auth.test.ts @@ -0,0 +1,300 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + createAnthropicOAuth, + createGitHubCopilotOAuth, + createGitHubCopilotTokenAuth, + createKimiCodingOAuth, + createOpenAiCodexOAuth, + createOpenRouterOAuth, + createRadiusOAuth, + createXaiOAuth, + type AuthEvent, + type AuthPrompt, + type Credential, + type ProviderAuthInteraction, +} from "../src/index.ts"; + +const now = () => 1_000_000; +const sleep = () => Promise.resolve(); + +function response(body: unknown, status = 200): Response { + return new Response(JSON.stringify(body), { + status, + headers: { "content-type": "application/json" }, + }); +} + +function interaction( + input: { answers?: readonly string[]; events?: AuthEvent[]; signal?: AbortSignal } = {}, +): ProviderAuthInteraction { + const answers = [...(input.answers ?? [])]; + return { + signal: input.signal ?? new AbortController().signal, + notify: (event) => input.events?.push(event), + prompt: (prompt: AuthPrompt) => { + const answer = answers.shift(); + if (answer !== undefined) return Promise.resolve(answer); + if (prompt.type === "manual_code") { + const state = input.events?.findLast((event) => event.type === "auth_url")?.url; + const value = state === undefined ? undefined : new URL(state).searchParams.get("state"); + return Promise.resolve(`http://localhost/callback?code=test-code&state=${value ?? ""}`); + } + return Promise.resolve(""); + }, + }; +} + +function oauth(value: Credential): asserts value is Extract { + assert.equal(value.type, "oauth"); +} + +function jwt(accountId: string): string { + const encode = (value: unknown) => Buffer.from(JSON.stringify(value)).toString("base64url"); + return `${encode({ alg: "none" })}.${encode({ + "https://api.openai.com/auth": { chatgpt_account_id: accountId }, + })}.signature`; +} + +test("Anthropic OAuth exchanges and refreshes without exposing tokens in events", async () => { + const events: AuthEvent[] = []; + const requests: { url: string; init?: RequestInit }[] = []; + const method = createAnthropicOAuth({ + now, + fetch: async (input, init) => { + requests.push({ url: String(input), ...(init === undefined ? {} : { init }) }); + return response({ + access_token: "anthropic-access", + refresh_token: "anthropic-refresh", + expires_in: 3600, + }); + }, + }); + + const credential = await method.login?.(interaction({ events })); + assert.ok(credential); + oauth(credential); + assert.equal(requests[0]?.url, "https://platform.claude.com/v1/oauth/token"); + assert.equal(credential.expiresAt, now() + 3_300_000); + assert.equal(JSON.stringify(events).includes("anthropic-access"), false); + assert.deepEqual(await method.toAuth(credential), { + headers: { + authorization: "Bearer anthropic-access", + "anthropic-beta": "claude-code-20250219,oauth-2025-04-20", + }, + }); + + const refreshed = await method.refresh(credential, new AbortController().signal); + assert.equal(refreshed.access, "anthropic-access"); + assert.equal(requests.length, 2); +}); + +test("OpenAI Codex device flow validates account identity before enabling auth", async () => { + const events: AuthEvent[] = []; + const token = jwt("account-123"); + const urls: string[] = []; + const method = createOpenAiCodexOAuth({ + now, + sleep, + fetch: async (input) => { + const url = String(input); + urls.push(url); + if (url.endsWith("/usercode")) { + return response({ device_auth_id: "device", user_code: "ABCD", interval: 0 }); + } + if (url.endsWith("/deviceauth/token")) { + return response({ authorization_code: "code", code_verifier: "verifier" }); + } + return response({ access_token: token, refresh_token: "codex-refresh", expires_in: 3600 }); + }, + }); + const credential = await method.login?.(interaction({ answers: ["device_code"], events })); + assert.ok(credential); + oauth(credential); + assert.equal(credential.metadata?.accountId, "account-123"); + assert.equal( + events.some((event) => event.type === "device_code"), + true, + ); + assert.equal(urls.at(-1), "https://auth.openai.com/oauth/token"); +}); + +test("OpenRouter OAuth persists the exchanged permanent key as an API key", async () => { + const events: AuthEvent[] = []; + const method = createOpenRouterOAuth({ + fetch: async () => response({ key: "openrouter-key" }), + }); + const credential = await method.login?.(interaction({ events })); + assert.deepEqual(credential, { type: "api_key", key: "openrouter-key" }); + assert.equal(JSON.stringify(events).includes("openrouter-key"), false); +}); + +test("Kimi and xAI device flows honor provider endpoints and refresh rotation", async () => { + for (const provider of ["kimi", "xai"] as const) { + const urls: string[] = []; + let tokenCalls = 0; + const fetchImpl: typeof fetch = async (input) => { + const url = String(input); + urls.push(url); + if (url.includes("device_authorization") || url.endsWith("/device/code")) { + return response({ + device_code: "device", + user_code: "CODE", + verification_uri: `https://${provider}.example/verify`, + verification_uri_complete: `https://${provider}.example/verify?code=CODE`, + interval: 1, + expires_in: 60, + }); + } + tokenCalls += 1; + return response({ + access_token: `${provider}-access-${tokenCalls}`, + refresh_token: `${provider}-refresh-${tokenCalls}`, + expires_in: 3600, + }); + }; + const method = + provider === "kimi" + ? createKimiCodingOAuth({ fetch: fetchImpl, now, sleep }) + : createXaiOAuth({ fetch: fetchImpl, now, sleep }); + const credential = await method.login?.(interaction({ events: [] })); + assert.ok(credential); + oauth(credential); + const refreshed = await method.refresh(credential, new AbortController().signal); + assert.equal(refreshed.access, `${provider}-access-2`); + assert.equal(urls.length, 3); + } +}); + +test("GitHub Copilot device flow preserves enterprise routing through refresh", async () => { + const calls: string[] = []; + let exchange = 0; + const method = createGitHubCopilotOAuth({ + now, + sleep, + fetch: async (input) => { + const url = String(input); + calls.push(url); + if (url.endsWith("/login/device/code")) { + return response({ + device_code: "device", + user_code: "CODE", + verification_uri: "https://github.example/device", + interval: 1, + expires_in: 60, + }); + } + if (url.endsWith("/login/oauth/access_token")) { + return response({ access_token: "github-token" }); + } + exchange += 1; + return response({ + token: `tid=x;proxy-ep=proxy.enterprise.example;value=${exchange}`, + expires_at: 5000, + }); + }, + }); + const credential = await method.login?.(interaction({ answers: ["github.example"], events: [] })); + assert.ok(credential); + oauth(credential); + assert.equal(credential.metadata?.domain, "github.example"); + assert.equal(credential.metadata?.baseUrl, "https://api.enterprise.example"); + const refreshed = await method.refresh(credential, new AbortController().signal); + assert.equal(refreshed.metadata?.domain, "github.example"); + assert.equal(calls.at(-1), "https://api.github.example/copilot_internal/v2/token"); +}); + +test("Copilot GitHub tokens exchange against the enterprise domain and derive routing", async () => { + const calls: string[] = []; + const method = createGitHubCopilotTokenAuth({ + fetch: async (input) => { + calls.push(String(input)); + return response({ + token: "tid=x;proxy-ep=proxy.business.example;value=token", + expires_at: 5000, + }); + }, + }); + const resolved = await method.resolve({ + context: { + env: (name) => + name === "COPILOT_GITHUB_TOKEN" + ? "github-access" + : name === "GITHUB_ENTERPRISE_URL" + ? "github.example" + : undefined, + fileExists: () => Promise.resolve(false), + }, + signal: new AbortController().signal, + }); + assert.equal(calls[0], "https://api.github.example/copilot_internal/v2/token"); + assert.equal(resolved?.auth.baseUrl, "https://api.business.example"); + assert.equal(resolved?.auth.apiKey?.includes("proxy-ep"), true); + assert.deepEqual(resolved?.secretValues, [ + "github-access", + "tid=x;proxy-ep=proxy.business.example;value=token", + ]); +}); + +test("Radius supports device authorization and refresh on the configured gateway", async () => { + const calls: string[] = []; + let tokens = 0; + const method = createRadiusOAuth("https://radius.example/base", { + now, + sleep, + fetch: async (input) => { + const url = String(input); + calls.push(url); + if (url.endsWith("/v1/oauth/device")) { + return response({ + device_code: "device", + user_code: "CODE", + verification_uri: "https://radius.example/verify", + interval: 1, + expires_in: 60, + }); + } + tokens += 1; + return response({ + access_token: `radius-access-${tokens}`, + refresh_token: "radius-refresh", + expires_in: 3600, + }); + }, + }); + const credential = await method.login?.(interaction({ answers: ["device_code"], events: [] })); + assert.ok(credential); + oauth(credential); + const refreshed = await method.refresh(credential, new AbortController().signal); + assert.equal(refreshed.access, "radius-access-2"); + assert.deepEqual(calls, [ + "https://radius.example/base/v1/oauth/device", + "https://radius.example/base/v1/oauth/token", + "https://radius.example/base/v1/oauth/token", + ]); +}); + +test("device OAuth cancellation stops before token persistence", async () => { + const controller = new AbortController(); + const method = createKimiCodingOAuth({ + sleep: async () => { + controller.abort(); + }, + fetch: async () => + response({ + device_code: "device", + user_code: "CODE", + verification_uri: "https://kimi.example/verify", + interval: 1, + expires_in: 60, + }), + }); + assert.ok(method.login); + await assert.rejects(method.login(interaction({ signal: controller.signal, events: [] })), { + name: "AbortError", + }); +}); diff --git a/packages/ai/test/transport-safety.test.ts b/packages/ai/test/transport-safety.test.ts new file mode 100644 index 0000000..8500586 --- /dev/null +++ b/packages/ai/test/transport-safety.test.ts @@ -0,0 +1,143 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { createServer } from "node:http"; +import { getDefaultAutoSelectFamily, setDefaultAutoSelectFamily } from "node:net"; +import test from "node:test"; + +import { + MAX_JSON_RESPONSE_BYTES, + readBoundedJson, + safeEndpoint, + safeFetch, + stripTrailingSlashes, +} from "../src/index.ts"; + +test("endpoint policy permits HTTPS and explicit loopback HTTP only", () => { + assert.equal( + safeEndpoint("https://example.com/v1///", { label: "test" }), + "https://example.com/v1", + ); + assert.equal( + safeEndpoint("http://127.0.0.1:11434/v1", { label: "test", allowLoopbackHttp: true }), + "http://127.0.0.1:11434/v1", + ); + for (const value of [ + "http://example.com/v1", + "https://10.0.0.1/v1", + "https://169.254.169.254/v1", + "https://[::1]/v1", + "https://user:password@example.com/v1", + "https://example.com/v1#fragment", + ]) + assert.throws(() => safeEndpoint(value, { label: "test" })); + assert.throws(() => + safeEndpoint("https://other.example/v1", { + label: "test", + expectedOrigin: "https://example.com/catalog", + }), + ); +}); + +test("trailing slash removal is linear and handles long non-matching input", () => { + const value = `${"/".repeat(100_000)}x`; + assert.equal(stripTrailingSlashes(value), value); + assert.equal(stripTrailingSlashes(`${value}///`), value); +}); + +test("safe transport rejects private DNS answers before dispatch", async () => { + let requests = 0; + await assert.rejects( + safeFetch( + "https://provider.example/v1/models", + { headers: { "x-api-key": "fake-secret" } }, + { + label: "test provider", + fetch: async () => { + requests += 1; + return new Response("{}"); + }, + resolve: () => Promise.resolve([{ address: "169.254.169.254", family: 4 }]), + }, + ), + /disallowed destination/, + ); + assert.equal(requests, 0); +}); + +test("safe transport rejects cross-origin redirects without forwarding request data", async () => { + const requests: { url: string; headers: Headers; body: unknown }[] = []; + await assert.rejects( + safeFetch( + "https://provider.example/v1/responses", + { + method: "POST", + headers: { "x-api-key": "fake-secret" }, + body: '{"prompt":"private prompt"}', + }, + { + label: "test provider", + fetch: async (input, init) => { + requests.push({ + url: String(input), + headers: new Headers(init?.headers), + body: init?.body, + }); + return new Response(null, { + status: 307, + headers: { location: "http://127.0.0.1/internal" }, + }); + }, + resolve: () => Promise.resolve([{ address: "8.8.8.8", family: 4 }]), + }, + ), + /must use HTTPS|unapproved origin/, + ); + assert.equal(requests.length, 1); + assert.equal(requests[0]?.url, "https://provider.example/v1/responses"); + assert.equal(requests[0]?.headers.get("x-api-key"), "fake-secret"); + assert.equal(requests[0]?.body, '{"prompt":"private prompt"}'); +}); + +test("bounded JSON rejects declared and chunked response overflow", async () => { + await assert.rejects( + readBoundedJson( + new Response("{}", { headers: { "content-length": String(MAX_JSON_RESPONSE_BYTES + 1) } }), + ), + /exceeds/, + ); + const response = new Response( + new ReadableStream({ + start(controller) { + controller.enqueue(new Uint8Array(Math.floor(MAX_JSON_RESPONSE_BYTES / 2))); + controller.enqueue(new Uint8Array(Math.floor(MAX_JSON_RESPONSE_BYTES / 2) + 1)); + controller.close(); + }, + }), + ); + await assert.rejects(readBoundedJson(response), /exceeds/); +}); + +test("real transport honors both Node DNS lookup callback shapes", async (context) => { + const previous = getDefaultAutoSelectFamily(); + context.after(() => setDefaultAutoSelectFamily(previous)); + const server = createServer((_request, response) => response.end("ok")); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + context.after(() => new Promise((resolve) => server.close(() => resolve()))); + const address = server.address(); + assert.ok(address && typeof address !== "string"); + for (const autoSelectFamily of [true, false]) { + setDefaultAutoSelectFamily(autoSelectFamily); + const response = await safeFetch( + `http://localhost:${address.port}/`, + {}, + { + label: "loopback transport test", + allowLoopbackHttp: true, + resolve: async () => [{ address: "127.0.0.1", family: 4 }], + }, + ); + assert.equal(await response.text(), "ok"); + } +}); diff --git a/packages/cli/README.md b/packages/cli/README.md index 08283c6..425e79f 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -18,6 +18,26 @@ Interactive sessions load global prompt templates from `~/.axl/prompts/*.md` and User themes load from `~/.axl/themes/*.json` and project overrides from `.axl/themes/*.json`. Select one with `/theme `. Existing theme directories are watched for live changes, and `/reload` rescans them. +## Model providers + +The daemon owns provider operations and canonical `{ providerId, modelId }` session selection. API dialect is model metadata and cannot be selected independently. The `provider` and `model` startup options set defaults for a new session only. + +```bash +axl providers +axl providers openrouter +axl models +axl models openrouter +axl login openrouter oauth +axl logout openrouter +axl refresh openrouter +``` + +`providers` checks explicit authentication status and shows safe source labels, login methods, catalog type, model count, and catalog errors. `models` groups text models by provider and includes unavailable reasons and published token prices. Neither command refreshes catalogs, and metadata listing does not read credentials or perform network requests. + +`login` supports `api_key` or `oauth` when offered by the provider. Prompt answers, API keys, OAuth codes, and tokens remain in the trusted daemon process-host adapter and never cross daemon RPC. Browser authorization is restricted to HTTPS URLs without embedded credentials. `logout` affects only the named provider. `refresh` is explicit and applies only to dynamic catalogs. Ctrl+C cancels an active provider operation, and externally effective operations are not replayed after reconnect. + +Errors include a safe category, provider and model identity when available, a concrete action, and retry guidance. There is no silent provider, model, dialect, authentication, or catalog fallback. + ## Print mode `axl print ` or `axl -p ` creates a durable session, runs one headless turn, writes only the final assistant text to stdout, and exits. Piped UTF-8 stdin is appended to the argument prompt after a blank line. Diagnostics and failures go to stderr, and a request for interactive input makes the command fail instead of waiting indefinitely. diff --git a/packages/cli/src/main.ts b/packages/cli/src/main.ts index 4df6896..2181439 100644 --- a/packages/cli/src/main.ts +++ b/packages/cli/src/main.ts @@ -12,15 +12,15 @@ import { homedir } from "node:os"; import { join } from "node:path"; import { TextDecoder } from "node:util"; -import type { AuthContext, CredentialStore } from "@axl/ai"; -import { AZURE_OPENAI_MODELS } from "@axl/ai/models"; +import type { CredentialStore } from "@axl/ai"; import { type CanonicalEvent, DEFAULT_MODEL_REQUEST_SETTINGS, - type ModelRequestSettings, - parseModelRequestSettings, encodeCanonicalEvent, MAX_WIRE_MESSAGE_BYTES, + type ModelRequestSettings, + type ProviderLoginMethod, + parseModelRequestSettings, type SessionProfile, type ThinkingLevel, } from "@axl/protocol"; @@ -31,18 +31,24 @@ import { type LocalSessionPlacement, listLocalSessions, localSandboxStateKey, + loginProviderFromTrustedHost, startLocalDaemon, } from "@axl/runtime"; import { type AxlClient, AxlClientError, subscribeSession } from "@axl/sdk"; import { connectUnixClient, createUnixDaemonHost } from "@axl/sdk/unix"; -import { azureLoginDialog, runAzureSetup } from "./azure-auth-ui.ts"; +import { createTerminalProviderLoginAdapter } from "./provider-auth-ui.ts"; +import { providerErrorMessage, runProviderCommand, usageLine } from "./provider-cli.ts"; import { loadTuiSettings, saveTuiSettings, type TuiSettings } from "./settings.ts"; const AXL_VERSION = process.env.AXL_BUILD_VERSION ?? "0.0.0-dev"; const HELP = `Usage: axl [session-id] [options] - axl login + axl providers [provider-id] + axl models [provider-id] + axl login [api_key|oauth] + axl logout + axl refresh [provider-id] axl doctor axl daemon [status|stop|restart] [options] axl print [prompt] [options] @@ -53,6 +59,7 @@ const HELP = `Usage: axl [session-id] [options] Options: --cwd Set the workspace directory + --provider Select the initial provider --model Select the initial model --thinking Select the initial reasoning effort --max-output-tokens Set an output ceiling or use the model maximum @@ -86,7 +93,11 @@ type SandboxChoice = "native" | "podman" | "docker"; interface CliArguments { command?: + | "providers" + | "models" | "login" + | "logout" + | "refresh" | "daemon" | "doctor" | "json" @@ -99,11 +110,14 @@ interface CliArguments { yes: boolean; force: boolean; sessionId?: string; + providerTarget?: string; + loginMethod?: ProviderLoginMethod; prompt: string[]; output?: string; raw: boolean; confirmPrefix: boolean; socket?: string; + provider?: string; model?: string; thinking?: ThinkingLevel; maxOutputTokens?: number | null; @@ -175,6 +189,7 @@ function parseArguments(argv: readonly string[]): CliArguments { else if (argument === "--yes") parsed.yes = true; else if (argument === "--force") parsed.force = true; else if (argument === "--socket") parsed.socket = next(); + else if (argument === "--provider") parsed.provider = next(); else if (argument === "--output") parsed.output = next(); else if (argument === "--raw") parsed.raw = true; else if (argument === "--confirm-prefix") parsed.confirmPrefix = true; @@ -238,16 +253,36 @@ function parseArguments(argv: readonly string[]): CliArguments { ) { parsed.prompt.push(argument); } else if ( + argument === "providers" || + argument === "models" || argument === "login" || + argument === "logout" || + argument === "refresh" || argument === "daemon" || argument === "doctor" || argument === "rpc" ) { parsed.command = argument; + } else if ( + !argument.startsWith("-") && + ["providers", "models", "login", "logout", "refresh"].includes(parsed.command ?? "") + ) { + if (parsed.providerTarget === undefined) parsed.providerTarget = argument; + else if ( + parsed.command === "login" && + parsed.loginMethod === undefined && + (argument === "api_key" || argument === "oauth") + ) { + parsed.loginMethod = argument; + } else throw new Error(`Unexpected ${parsed.command} argument ${argument}`); } else if (!argument.startsWith("-") && !parsed.command?.startsWith("session-")) { parsed.sessionId = argument; } else throw new Error(`Unknown argument ${argument}`); } + if (parsed.command === "login" || parsed.command === "logout") { + if (parsed.providerTarget === undefined) + throw new Error(`${parsed.command} requires a provider ID`); + } if ( (parsed.interrupt || parsed.yes || parsed.force) && parsed.daemonAction !== "stop" && @@ -346,39 +381,8 @@ function samePlacement(left: LocalSessionPlacement, right: LocalSessionPlacement return left.engine === right.engine && left.image === right.image; } -async function ensureCredentials( - store: CredentialStore, - allowInteractiveSetup = true, -): Promise { - const { - AuthError, - AZURE_OPENAI_PROVIDER_ID, - azureOpenAiAuthMethod, - nodeAuthContext, - resolveProviderAuth, - } = await import("@axl/ai"); - try { - await resolveProviderAuth( - AZURE_OPENAI_PROVIDER_ID, - { apiKey: azureOpenAiAuthMethod }, - store, - nodeAuthContext, - ); - } catch (error) { - if ( - error instanceof AuthError && - error.code === "not_configured" && - allowInteractiveSetup && - process.stdin.isTTY - ) { - await runAzureSetup(process.stdin, process.stdout, store, nodeAuthContext); - return; - } - throw error; - } -} - interface ActiveConfig { + readonly providerId: string; readonly requestSettings: ModelRequestSettings; readonly modelId: string; readonly thinkingLevel: ThinkingLevel; @@ -456,6 +460,7 @@ async function connectExpectedDaemon( async function connectOrStartDaemon(input: { readonly requestSettings: ModelRequestSettings; readonly socketPath: string; + readonly provider: string; readonly model: string; readonly thinking: ThinkingLevel; readonly unsafe: boolean; @@ -485,6 +490,8 @@ async function connectOrStartDaemon(input: { "daemon", "--socket", input.socketPath, + "--provider", + input.provider, "--model", input.model, "--thinking", @@ -610,6 +617,7 @@ async function runHeadless( ): Promise { const opened = await client.request("session.create", { cwd: input.cwd, + providerId: input.active.providerId, requestSettings: input.active.requestSettings, modelId: input.active.modelId, thinkingLevel: input.active.thinkingLevel, @@ -695,6 +703,9 @@ async function runPrint(client: AxlClient, input: HeadlessInput): Promise .map((content) => content.text) .join(""); process.stdout.write(text.endsWith("\n") ? text : `${text}\n`); + if (terminal.payload.usage !== undefined) { + process.stderr.write(`${usageLine(terminal.payload.usage)}\n`); + } } async function writeJsonEvent(event: CanonicalEvent): Promise { @@ -769,6 +780,10 @@ async function main(): Promise { if (cli.profile !== undefined && cli.resume) { throw new Error("--profile cannot be combined with --resume"); } + if (cli.command === "login") { + const { assertInteractiveTerminal } = await import("@axl/tui"); + assertInteractiveTerminal(process.stdin, process.stdout); + } const headlessMode = cli.command === "json" || cli.command === "print" ? cli.command : undefined; const headlessPrompt = @@ -860,26 +875,16 @@ async function main(): Promise { let settings = await loadTuiSettings(settingsPath); timing.mark("settings"); - let credentialsPromise: Promise<{ store: CredentialStore; context: AuthContext }> | undefined; + let credentialsPromise: Promise<{ store: CredentialStore }> | undefined; const credentials = () => { - credentialsPromise ??= import("@axl/ai").then(({ FileCredentialStore, nodeAuthContext }) => ({ + credentialsPromise ??= import("@axl/ai").then(({ FileCredentialStore }) => ({ store: new FileCredentialStore(join(axlHome, "credentials.json")), - context: nodeAuthContext, })); return credentialsPromise; }; - if (cli.command === "login") { - const [{ store, context }, { assertInteractiveTerminal }] = await Promise.all([ - credentials(), - import("@axl/tui"), - ]); - assertInteractiveTerminal(process.stdin, process.stdout); - await runAzureSetup(process.stdin, process.stdout, store, context); - process.exit(0); - } - const active: ActiveConfig = { + providerId: cli.provider ?? settings.providerId ?? "azure-openai-responses", requestSettings: parseModelRequestSettings({ maxOutputTokens: cli.maxOutputTokens === undefined @@ -903,7 +908,6 @@ async function main(): Promise { } if (cli.command === "daemon" && cli.daemonAction === undefined) { const { store } = await credentials(); - await ensureCredentials(store); const daemon = await startLocalDaemon({ buildVersion: AXL_VERSION, onStopped: () => process.exit(0), @@ -929,12 +933,31 @@ async function main(): Promise { return; } + const loginFromThisHost = async ( + providerId: string, + method: ProviderLoginMethod, + signal?: AbortSignal, + presentation?: import("@axl/tui").ProviderLoginPresentation, + ) => { + const { store } = await credentials(); + return loginProviderFromTrustedHost({ + store, + axlHome, + adapter: createTerminalProviderLoginAdapter(process.stdin, process.stdout, presentation), + providerId, + method, + ...(signal === undefined ? {} : { signal }), + }); + }; + const clientKind = cli.command === "json" || cli.command === "print" ? cli.command : cli.command === "rpc" ? "rpc_probe" - : "tui"; + : ["providers", "models", "login", "logout", "refresh"].includes(cli.command ?? "") + ? "cli" + : "tui"; const connectTarget = async (target: LocalDaemonTarget): Promise => { await mkdir(target.stateDirectory, { recursive: true, mode: 0o700 }); try { @@ -946,12 +969,11 @@ async function main(): Promise { clientKind, ); } catch (error) { - if (!missingDaemon(error)) throw error; - const { store } = await credentials(); - await ensureCredentials(store, cli.command === undefined); + if (error instanceof SecurityModeMismatchError || !missingDaemon(error)) throw error; return connectOrStartDaemon({ requestSettings: active.requestSettings, socketPath: target.socketPath, + provider: active.providerId, model: active.modelId, thinking: active.thinkingLevel, unsafe: target.unsafe, @@ -987,6 +1009,21 @@ async function main(): Promise { await bridgeRpc(socketPath); return; } + if (["providers", "models", "login", "logout", "refresh"].includes(cli.command ?? "")) { + try { + await runProviderCommand({ + client, + command: cli.command as "providers" | "models" | "login" | "logout" | "refresh", + ...(cli.providerTarget === undefined ? {} : { providerId: cli.providerTarget }), + ...(cli.loginMethod === undefined ? {} : { loginMethod: cli.loginMethod }), + ...(cli.command === "login" ? { login: loginFromThisHost } : {}), + write: (value) => process.stdout.write(value), + }); + } finally { + client.close(); + } + return; + } if (cli.command === "json" || cli.command === "print") { if (headlessPrompt === undefined) throw new Error("Headless prompt was not loaded"); const input = { @@ -1089,19 +1126,16 @@ async function main(): Promise { ], clearStartupLine: startupIndicator, reconnectClient: () => connectTarget(currentTarget), + loginProvider: (providerId, method, signal, presentation) => + loginFromThisHost(providerId, method, signal, presentation), onPreferenceChange: persistSettings, - models: AZURE_OPENAI_MODELS.map((model) => model.modelId), - modelCatalog: AZURE_OPENAI_MODELS, + currentProvider: active.providerId, requestSettings: active.requestSettings, currentModel: active.modelId, currentThinking: active.thinkingLevel, ...(cli.profile === undefined ? {} : { profile: cli.profile }), webFetch: active.webFetch, webSearch: active.webSearch, - loadLogin: async () => { - const { store, context } = await credentials(); - return azureLoginDialog(store, context); - }, ...(cli.sessionId === undefined ? {} : { sessionId: cli.sessionId }), onExit: () => { void settingsWrite.finally(() => process.exit(0)); @@ -1114,7 +1148,7 @@ async function main(): Promise { main().catch((error: unknown) => { if (process.stdout.isTTY) process.stdout.write("\r\x1b[2K"); - process.stderr.write(`axl: ${error instanceof Error ? error.message : String(error)}\n`); + process.stderr.write(`axl: ${providerErrorMessage(error)}\n`); process.exit( error instanceof AxlClientError && ["busy", "confirmation_required", "state_changed"].includes(error.code) diff --git a/packages/cli/src/provider-auth-ui.ts b/packages/cli/src/provider-auth-ui.ts new file mode 100644 index 0000000..d6aee5b --- /dev/null +++ b/packages/cli/src/provider-auth-ui.ts @@ -0,0 +1,174 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { spawn } from "node:child_process"; + +import type { AuthEvent, AuthPrompt } from "@axl/ai"; +import type { TrustedProviderLoginAdapter } from "@axl/runtime"; +import type { ProviderLoginPresentation } from "@axl/tui"; +import { promptLine, type SetupInput, type SetupOutput, sanitizeTerminalText } from "@axl/tui"; + +export function validatedAuthorizationUrl(value: string): URL { + let url: URL; + try { + url = new URL(value); + } catch (cause) { + throw new Error("Provider supplied an invalid authorization URL", { cause }); + } + if (url.protocol !== "https:") { + throw new Error("Provider authorization URLs must use HTTPS"); + } + if (url.username || url.password) { + throw new Error("Provider authorization URLs must not contain credentials"); + } + return url; +} + +interface BrowserProcess { + once(event: "error", listener: (error: Error) => void): unknown; + unref(): void; +} + +type BrowserLauncher = (file: string, args: readonly string[]) => BrowserProcess; + +export function openAuthorizationUrl( + url: URL, + output: SetupOutput, + launch: BrowserLauncher = (file, args) => spawn(file, args, { detached: true, stdio: "ignore" }), +): void { + const command = + process.platform === "darwin" + ? { file: "open", args: [url.href] } + : process.platform === "win32" + ? { file: "rundll32", args: ["url.dll,FileProtocolHandler", url.href] } + : { file: "xdg-open", args: [url.href] }; + const child = launch(command.file, command.args); + child.once("error", (error) => { + output.write(` Could not open the authorization URL automatically: ${safe(error.message)}\n`); + }); + child.unref(); +} + +function safe(value: string): string { + return sanitizeTerminalText(value).replace(/\s+/gu, " ").trim(); +} + +async function answerPrompt( + input: SetupInput, + output: SetupOutput, + prompt: AuthPrompt, + signal: AbortSignal, +): Promise { + signal.throwIfAborted(); + if (prompt.type === "select") { + output.write(`${safe(prompt.message)}\n`); + prompt.options.forEach((option, index) => { + const description = option.description ? `: ${safe(option.description)}` : ""; + output.write(` ${index + 1}. ${safe(option.label)}${description}\n`); + }); + while (true) { + const answer = await promptLine(input, output, " Select: ", { signal }); + const numeric = Number(answer); + const selected = Number.isSafeInteger(numeric) ? prompt.options[numeric - 1] : undefined; + const byId = prompt.options.find((option) => option.id === answer); + if (selected !== undefined || byId !== undefined) return (selected ?? byId)?.id as string; + output.write(" Choose one of the listed options.\n"); + } + } + return promptLine(input, output, ` ${safe(prompt.message)}: `, { + mask: prompt.type === "secret" || prompt.type === "manual_code", + signal: + prompt.type === "manual_code" && prompt.signal + ? AbortSignal.any([signal, prompt.signal]) + : signal, + allowEmpty: prompt.type === "text", + }); +} + +function presentEvent(output: SetupOutput, event: AuthEvent): void { + if (event.type === "state") return; + if (event.type === "auth_url") { + const url = validatedAuthorizationUrl(event.url); + output.write(` ${safe(event.instructions ?? "Complete authorization in your browser.")}\n`); + output.write(` ${url.href}\n`); + openAuthorizationUrl(url, output); + return; + } + if (event.type === "device_code") { + const url = validatedAuthorizationUrl(event.verificationUri); + output.write(` Open ${url.href}\n Code: ${safe(event.userCode)}\n`); + openAuthorizationUrl(url, output); + return; + } + output.write(` ${safe(event.message)}\n`); + if (event.type === "info") { + for (const link of event.links ?? []) { + const url = validatedAuthorizationUrl(link.url); + output.write(` ${safe(link.label ?? "Open")}: ${url.href}\n`); + } + } +} + +/** Keeps provider prompts and answers inside the invoking trusted process host. */ +export function createTerminalProviderLoginAdapter( + input: SetupInput, + output: SetupOutput, + presentation?: ProviderLoginPresentation, +): TrustedProviderLoginAdapter { + return { + createInteraction: ({ signal }) => { + if (input.isTTY !== true || output.isTTY !== true) { + throw new Error( + "Interactive provider login requires a terminal attached to the invoking host", + ); + } + return { + signal, + prompt: (prompt) => + presentation === undefined + ? answerPrompt(input, output, prompt, signal) + : presentation.prompt({ + message: prompt.message, + ...(prompt.type === "select" + ? { + options: prompt.options.map((option) => ({ + value: option.id, + label: option.label, + ...(option.description === undefined + ? {} + : { description: option.description }), + })), + } + : { + ...(prompt.placeholder === undefined + ? {} + : { placeholder: prompt.placeholder }), + mask: prompt.type === "secret" || prompt.type === "manual_code", + allowEmpty: prompt.type === "text", + ...(prompt.type === "manual_code" && prompt.signal + ? { signal: prompt.signal } + : {}), + }), + }), + notify: (event) => { + if (presentation === undefined) { + presentEvent(output, event); + return; + } + if (event.type === "state") return; + // Browser opening stays in the host; only safe presentation text enters the TUI. + const lines: string[] = []; + presentEvent( + { + write: (text) => { + lines.push(text); + presentation.notify(lines.join("").trim()); + }, + }, + event, + ); + }, + }; + }, + }; +} diff --git a/packages/cli/src/provider-cli.ts b/packages/cli/src/provider-cli.ts new file mode 100644 index 0000000..43006ab --- /dev/null +++ b/packages/cli/src/provider-cli.ts @@ -0,0 +1,161 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { stripVTControlCharacters } from "node:util"; + +import type { + ProviderAuthenticationStatus, + ProviderInventoryGroup, + ProviderLoginMethod, + Usage, +} from "@axl/protocol"; +import { type AxlClient, ProviderClientError } from "@axl/sdk"; + +function safe(value: string): string { + const text = [...stripVTControlCharacters(value).replace(/\s+/gu, " ")] + .filter((character) => { + const code = character.codePointAt(0) ?? 0; + return !( + code < 32 || + (code >= 127 && code <= 159) || + (code >= 0x202a && code <= 0x202e) || + (code >= 0x2066 && code <= 0x2069) + ); + }) + .join(""); + return text.trim(); +} + +export function providerErrorMessage(error: unknown): string { + if (!(error instanceof ProviderClientError)) { + return error instanceof Error ? safe(error.message) : "provider operation failed"; + } + const subject = [error.details.providerId, error.details.modelId].filter(Boolean).join("/"); + const action = error.details.action.replaceAll("_", " "); + return [ + safe(error.message), + subject ? `${error.details.category}: ${subject}` : error.details.category, + `action: ${action}`, + ...(error.retryable ? ["retryable"] : []), + ].join(" · "); +} + +function authenticationLabel(status: ProviderAuthenticationStatus): string { + return [status.phase.replaceAll("_", " "), status.method, status.source] + .filter(Boolean) + .join(" · "); +} + +export function providerStatusLines(providers: readonly ProviderInventoryGroup[]): string[] { + return providers.flatMap((provider) => [ + `${provider.displayName} (${provider.providerId})`, + ` authentication ${authenticationLabel(provider.authentication)}`, + ` login methods ${provider.loginMethods.join(", ") || "none"}`, + ` catalog ${provider.catalog.refreshable ? "dynamic" : "static"} · ${provider.models.length} text models`, + ...(provider.catalogError === undefined + ? [] + : [ + ` catalog error ${safe(provider.catalogError.message)} · action: ${provider.catalogError.action.replaceAll("_", " ")}`, + ]), + ]); +} + +export function modelLines(providers: readonly ProviderInventoryGroup[]): string[] { + return providers.flatMap((provider) => [ + `${provider.displayName} (${provider.providerId})`, + ...provider.models.map((model) => { + const unavailable = + model.availability.status === "available" + ? "" + : ` · ${model.availability.status}${model.availability.reason ? `: ${safe(model.availability.reason)}` : ""}`; + const cost = model.cost + ? ` · $${model.cost.inputUsdPerMTok}/$${model.cost.outputUsdPerMTok} per MTok` + : ""; + return ` ${model.modelId} · ${model.apiDialect}${cost}${unavailable}`; + }), + ]); +} + +export function usageLine(usage: Usage): string { + const values = [ + `input ${usage.inputTokens}`, + `output ${usage.outputTokens}`, + `cache read ${usage.cacheReadTokens}`, + `cache write ${usage.cacheWriteTokens}`, + ...(usage.reasoningTokens === undefined ? [] : [`reasoning ${usage.reasoningTokens}`]), + ...(usage.costUsd === undefined ? [] : [`cost $${usage.costUsd.toFixed(6)}`]), + ]; + return `usage: ${values.join(" · ")}`; +} + +export async function runProviderCommand(input: { + readonly client: AxlClient; + readonly command: "providers" | "models" | "login" | "logout" | "refresh"; + readonly providerId?: string; + readonly loginMethod?: ProviderLoginMethod; + readonly login?: ( + providerId: string, + method: ProviderLoginMethod, + ) => Promise; + readonly write: (value: string) => void; +}): Promise { + const params = input.providerId === undefined ? {} : { providerId: input.providerId }; + if (input.command === "providers") { + const listed = await input.client.listProviders(params); + const statuses = await input.client.providerAuthenticationStatus(params); + const statusByProvider = new Map( + statuses.providers.map((status) => [status.providerId, status]), + ); + const providers = listed.providers.map((provider) => ({ + ...provider, + authentication: statusByProvider.get(provider.providerId) ?? provider.authentication, + })); + input.write(`${providerStatusLines(providers).join("\n")}\n`); + return; + } + if (input.command === "models") { + input.write(`${modelLines((await input.client.listProviders(params)).providers).join("\n")}\n`); + return; + } + if (input.command === "refresh") { + const refreshed = await input.client.refreshProviderCatalogs(params); + input.write( + `${refreshed.providers + .map((provider) => + provider.error === undefined + ? `${provider.providerId}: ${provider.status} · ${provider.modelCount} models` + : `${provider.providerId}: ${provider.status} · ${safe(provider.error.message)} · action: ${provider.error.action.replaceAll("_", " ")}`, + ) + .join("\n")}\n`, + ); + if (refreshed.providers.some((provider) => provider.status === "failed")) { + throw new Error("One or more provider catalogs failed to refresh"); + } + return; + } + if (input.providerId === undefined) throw new Error(`${input.command} requires a provider ID`); + if (input.command === "logout") { + const status = await input.client.logoutProvider({ providerId: input.providerId }); + input.write(`${status.providerId}: ${authenticationLabel(status)}\n`); + return; + } + const listed = await input.client.listProviders({ providerId: input.providerId }); + const provider = listed.providers[0]; + if (provider === undefined) throw new Error(`Unknown provider ${input.providerId}`); + const method = + input.loginMethod ?? + (provider.loginMethods.length === 1 ? provider.loginMethods[0] : undefined); + if (method === undefined) { + throw new Error( + `Provider ${input.providerId} requires a login method: ${provider.loginMethods.join(" or ") || "none available"}`, + ); + } + if (!provider.loginMethods.includes(method)) { + throw new Error(`Provider ${input.providerId} does not support ${method} login`); + } + const status = + input.login === undefined + ? await input.client.loginProvider({ providerId: input.providerId, method }) + : await input.login(input.providerId, method); + input.write(`${status.providerId}: ${authenticationLabel(status)}\n`); +} diff --git a/packages/cli/src/settings.ts b/packages/cli/src/settings.ts index c16aa5b..3163bf0 100644 --- a/packages/cli/src/settings.ts +++ b/packages/cli/src/settings.ts @@ -29,6 +29,7 @@ export interface AxlSettings { export interface TuiSettings { readonly requestSettings?: ModelRequestSettings; readonly version: 1; + readonly providerId?: string; readonly modelId?: string; readonly thinkingLevel?: ThinkingLevel; readonly theme?: string; @@ -91,6 +92,7 @@ function parseSettings(value: unknown, path: string): TuiSettings { } const allowed = new Set([ "version", + "providerId", "modelId", "thinkingLevel", "requestSettings", @@ -118,8 +120,10 @@ function parseSettings(value: unknown, path: string): TuiSettings { if (input.requestSettings !== undefined) parseModelRequestSettings(input.requestSettings, `${path}.requestSettings`); if (input.version !== 1) throw new Error(`${path}: version must be 1`); - if (input.modelId !== undefined && (typeof input.modelId !== "string" || !input.modelId)) { - throw new Error(`${path}: modelId must be a non-empty string`); + for (const field of ["providerId", "modelId"] as const) { + if (input[field] !== undefined && (typeof input[field] !== "string" || !input[field])) { + throw new Error(`${path}: ${field} must be a non-empty string`); + } } if ( input.thinkingLevel !== undefined && diff --git a/packages/cli/test/provider-cli.test.ts b/packages/cli/test/provider-cli.test.ts new file mode 100644 index 0000000..f85f377 --- /dev/null +++ b/packages/cli/test/provider-cli.test.ts @@ -0,0 +1,203 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { PassThrough } from "node:stream"; +import test from "node:test"; + +import type { AxlClient } from "@axl/sdk"; + +import { + createTerminalProviderLoginAdapter, + openAuthorizationUrl, + validatedAuthorizationUrl, +} from "../src/provider-auth-ui.ts"; +import { providerErrorMessage, runProviderCommand, usageLine } from "../src/provider-cli.ts"; + +class Input extends PassThrough { + isTTY = true; + isRaw = false; + + setRawMode(value: boolean): this { + this.isRaw = value; + return this; + } +} + +function client(): AxlClient { + return { + listProviders: () => + Promise.resolve({ + providers: [ + { + providerId: "test-provider", + displayName: "Test Provider", + enabled: true, + authMethods: ["environment"], + loginMethods: ["api_key"], + authentication: { providerId: "test-provider", phase: "idle" }, + catalog: { refreshable: true }, + models: [ + { + providerId: "test-provider", + modelId: "test-model", + displayName: "Test Model", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: false }, + reasoning: false, + supportedThinkingLevels: ["off"], + contextWindow: 8_000, + maxOutputTokens: 1_000, + availability: { status: "unavailable", reason: "configure a region" }, + }, + ], + }, + ], + }), + providerAuthenticationStatus: () => + Promise.resolve({ + providers: [ + { + providerId: "test-provider", + phase: "authenticated", + method: "api_key", + source: "TEST_API_KEY", + }, + ], + }), + refreshProviderCatalogs: () => + Promise.resolve({ + providers: [{ providerId: "test-provider", status: "refreshed", modelCount: 1 }], + }), + loginProvider: ({ providerId, method }: { providerId: string; method: "api_key" | "oauth" }) => + Promise.resolve({ providerId, phase: "authenticated", method }), + logoutProvider: ({ providerId }: { providerId: string }) => + Promise.resolve({ providerId, phase: "logged_out" }), + } as unknown as AxlClient; +} + +test("provider CLI commands render grouped safe status and model data", async () => { + const output: string[] = []; + const write = (value: string) => output.push(value); + const sdk = client(); + await runProviderCommand({ client: sdk, command: "providers", write }); + await runProviderCommand({ client: sdk, command: "models", write }); + await runProviderCommand({ client: sdk, command: "refresh", write }); + let hostLogins = 0; + await runProviderCommand({ + client: sdk, + command: "login", + providerId: "test-provider", + loginMethod: "api_key", + login: (providerId, method) => { + hostLogins += 1; + return Promise.resolve({ providerId, phase: "authenticated", method }); + }, + write, + }); + await runProviderCommand({ client: sdk, command: "logout", providerId: "test-provider", write }); + + const rendered = output.join(""); + assert.match(rendered, /Test Provider \(test-provider\)/); + assert.match(rendered, /authenticated · api_key · TEST_API_KEY/); + assert.match(rendered, /unavailable: configure a region/); + assert.match(rendered, /test-provider: refreshed · 1 models/); + assert.equal(hostLogins, 1); +}); + +test("trusted terminal adapter masks and cancels prompt answers without retaining raw mode", async () => { + const input = new Input(); + let output = ""; + const adapter = createTerminalProviderLoginAdapter(input, { + isTTY: true, + write: (value) => { + output += value; + }, + }); + const controller = new AbortController(); + const interaction = adapter.createInteraction({ + providerId: "test-provider", + method: "api_key", + signal: controller.signal, + }); + const answer = interaction.prompt({ type: "secret", message: "API key" }); + input.write("super-secret\r"); + assert.equal(await answer, "super-secret"); + assert.equal(output.includes("super-secret"), false); + assert.match(output, /\*{12}/); + assert.equal(input.isRaw, false); + + const cancelled = interaction.prompt({ type: "text", message: "Account" }); + controller.abort(); + await assert.rejects(cancelled, /Setup aborted/); + assert.equal(input.isRaw, false); +}); + +test("browser launch failures remain visible", () => { + let output = ""; + let unrefCalled = false; + openAuthorizationUrl( + validatedAuthorizationUrl("https://example.com/login"), + { + write: (value) => { + output += value; + }, + }, + () => ({ + once: (_event, listener) => listener(new Error("launcher unavailable\nretry manually")), + unref: () => { + unrefCalled = true; + }, + }), + ); + assert.match(output, /Could not open the authorization URL automatically/); + assert.match(output, /launcher unavailable retry manually/); + assert.equal(unrefCalled, true); +}); + +test("authorization URLs are restricted and usage remains explicit", () => { + assert.equal(validatedAuthorizationUrl("https://example.com/login").hostname, "example.com"); + assert.throws(() => validatedAuthorizationUrl("http://example.com/login"), /must use HTTPS/); + assert.throws( + () => validatedAuthorizationUrl("https://user:pass@example.com/login"), + /must not contain credentials/, + ); + assert.equal( + usageLine({ + inputTokens: 10, + outputTokens: 4, + cacheReadTokens: 3, + cacheWriteTokens: 2, + reasoningTokens: 1, + costUsd: 0.000123, + }), + "usage: input 10 · output 4 · cache read 3 · cache write 2 · reasoning 1 · cost $0.000123", + ); + assert.equal(providerErrorMessage(new Error("safe failure\nnext")), "safe failure next"); +}); + +test("aggregate refresh failures reject after rendering actionable results", async () => { + const sdk = client(); + sdk.refreshProviderCatalogs = async () => ({ + providers: [ + { + providerId: "test-provider", + status: "failed", + modelCount: 0, + error: { code: "catalog_refresh_failed", message: "Service unavailable", action: "retry" }, + }, + ], + }); + let output = ""; + await assert.rejects( + runProviderCommand({ + client: sdk, + command: "refresh", + write: (value) => { + output += value; + }, + }), + /failed to refresh/, + ); + assert.match(output, /Service unavailable/); +}); diff --git a/packages/cli/test/unsafe-cli.test.ts b/packages/cli/test/unsafe-cli.test.ts index 265653e..3838264 100644 --- a/packages/cli/test/unsafe-cli.test.ts +++ b/packages/cli/test/unsafe-cli.test.ts @@ -8,6 +8,7 @@ import assert from "node:assert/strict"; import { type ChildProcess, spawn, spawnSync } from "node:child_process"; import { mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises"; +import { createServer } from "node:http"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test, { type TestContext } from "node:test"; @@ -31,6 +32,11 @@ test("--help and --version do not require credentials", () => { assert.match(help.stdout, /axl print/); assert.match(help.stdout, /axl json/); assert.match(help.stdout, /axl rpc/); + assert.match(help.stdout, /axl providers/); + assert.match(help.stdout, /axl models/); + assert.match(help.stdout, /axl login /); + assert.match(help.stdout, /axl logout /); + assert.match(help.stdout, /axl refresh/); const version = spawnSync(process.execPath, [entry, "--version"], { encoding: "utf8" }); assert.equal(version.status, 0); @@ -340,7 +346,11 @@ test("print and JSON run one headless turn", async (context) => { { ...process.env, HOME: directory }, "piped input\n", ); - assert.deepEqual(result, { code: 0, stdout: "printed response\n", stderr: "" }); + assert.deepEqual(result, { + code: 0, + stdout: "printed response\n", + stderr: "usage: input 1 · output 2 · cache read 0 · cache write 0\n", + }); assert.equal(prompt, "Summarize this\n\npiped input\n"); const json = await runCli( @@ -515,3 +525,77 @@ test("clients refuse a different OCI engine or image", async (context) => { assert.match(result.stderr, /Daemon security mode is sandboxed\/podman/); } }); + +test("built CLI discovers and dispatches a named models.json provider over loopback", async (context) => { + const home = await temporaryDirectory(context); + const workspace = join(home, "workspace"); + await mkdir(workspace); + await mkdir(join(home, ".axl")); + let requests = 0; + const server = createServer((request, response) => { + requests++; + assert.equal(request.url, "/v1/chat/completions"); + assert.equal(request.headers.authorization, undefined); + request.resume(); + response.writeHead(200, { "content-type": "text/event-stream" }); + response.end( + 'data: {"choices":[{"index":0,"delta":{"content":"LOCAL_SMOKE_OK"},"finish_reason":null}]}\n\ndata: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":1,"completion_tokens":1}}\n\ndata: [DONE]\n\n', + ); + }); + await new Promise((resolve) => server.listen(0, "127.0.0.1", resolve)); + context.after(() => new Promise((resolve) => server.close(() => resolve()))); + const address = server.address(); + assert.ok(address && typeof address !== "string"); + await writeFile( + join(home, ".axl", "models.json"), + JSON.stringify({ + providers: { + local: { + baseUrl: `http://127.0.0.1:${address.port}/v1`, + models: [ + { + modelId: "echo", + displayName: "Local echo", + apiDialect: "openai-chat", + compatibility: { dialect: "openai-chat", supportsDeveloperRole: false }, + capabilities: { toolUse: true, structuredOutput: false, imageInput: false }, + reasoning: false, + contextWindow: 8192, + maxOutputTokens: 1024, + }, + ], + }, + }, + }), + ); + const env = { HOME: home, PATH: process.env.PATH }; + const child = spawn(process.execPath, [entry, "daemon", "--unsafe"], { env, stdio: "ignore" }); + context.after(() => stopChild(child)); + const client = await connectEventually(join(home, ".axl", "unsafe", "axl.sock"), child); + await client.close(); + const listed = await runCli(["models", "local", "--unsafe"], env); + assert.equal(listed.code, 0, listed.stderr); + assert.match(listed.stdout, /Local echo|echo/); + assert.equal(requests, 0); + const result = await runCli( + [ + "print", + "Reply ok", + "--unsafe", + "--cwd", + workspace, + "--provider", + "local", + "--model", + "echo", + "--thinking", + "off", + "--profile", + "exec", + ], + env, + ); + assert.equal(result.code, 0, result.stderr); + assert.match(result.stdout, /LOCAL_SMOKE_OK/); + assert.equal(requests, 1); +}); diff --git a/packages/daemon/src/daemon.ts b/packages/daemon/src/daemon.ts index 8d676f6..6b27feb 100644 --- a/packages/daemon/src/daemon.ts +++ b/packages/daemon/src/daemon.ts @@ -52,6 +52,7 @@ import { import { type CommandAcceptance, CommandJournal, CommandJournalError } from "./command-journal.ts"; import { DataDirectoryLock } from "./data-directory-lock.ts"; +import type { ProviderManagementService } from "./provider-management.ts"; import { DaemonError, SessionManager, type SessionManagerOptions } from "./session-manager.ts"; export type DaemonSecurityMode = "sandboxed" | "unsafe"; @@ -69,6 +70,7 @@ export interface DaemonOptions extends SessionManagerOptions { readonly cursorLifetimeMs?: number; readonly heartbeatIntervalMs?: number; readonly presenceTimeoutMs?: number; + readonly providerManagement?: ProviderManagementService; } const MAX_PENDING_REQUESTS = 64; @@ -203,6 +205,8 @@ export class AxlDaemon { private readonly cursorLifetimeMs: number; private readonly heartbeatIntervalMs: number; private readonly presenceTimeoutMs: number; + private readonly providerManagement: ProviderManagementService | undefined; + private readonly capabilities: readonly string[]; private readonly hostOptions: Pick< DaemonOptions, "buildVersion" | "onStopped" | "forceTerminate" @@ -235,6 +239,11 @@ export class AxlDaemon { this.cursorLifetimeMs = options.cursorLifetimeMs ?? 300_000; this.heartbeatIntervalMs = options.heartbeatIntervalMs ?? HEARTBEAT_INTERVAL_MS; this.presenceTimeoutMs = options.presenceTimeoutMs ?? PRESENCE_TIMEOUT_MS; + this.providerManagement = options.providerManagement; + this.capabilities = + this.providerManagement === undefined + ? WIRE_CAPABILITIES.filter((capability) => !capability.startsWith("provider.")) + : WIRE_CAPABILITIES; if ( !Number.isSafeInteger(this.snapshotIdleLifetimeMs) || this.snapshotIdleLifetimeMs <= 0 || @@ -333,6 +342,7 @@ export class AxlDaemon { // Every request admitted before the gate must finish its journal outcome first. await Promise.all([...this.pending]); await this.sessions.disposeAll(); + await this.providerManagement?.dispose?.(); await this.dataLock?.release({ allowMissing: true }); this.dataLock = undefined; await this.removeOwnedSocket(); @@ -514,7 +524,7 @@ export class AxlDaemon { kind: "hello", wireVersion: WIRE_PROTOCOL_VERSION, daemonInstanceId: this.daemonInstanceId, - capabilities: WIRE_CAPABILITIES, + capabilities: this.capabilities, limits: { maxMessageBytes: MAX_WIRE_MESSAGE_BYTES, maxPendingRequests: MAX_PENDING_REQUESTS, @@ -765,7 +775,7 @@ export class AxlDaemon { state.lastSeenAt = now; state.grantedCapabilities = new Set( request.params.requestedCapabilities.filter((capability) => - WIRE_CAPABILITIES.includes(capability as (typeof WIRE_CAPABILITIES)[number]), + this.capabilities.includes(capability), ), ); } else { @@ -781,6 +791,11 @@ export class AxlDaemon { } } const cancellable = + request.method === "provider.list" || + request.method === "provider.catalog.refresh" || + request.method === "provider.auth.status" || + request.method === "provider.auth.login" || + request.method === "provider.auth.logout" || request.method === "session.history" || request.method === "session.workspace.list" || request.method === "session.workspace.read" || @@ -816,7 +831,9 @@ export class AxlDaemon { ? error.code : error instanceof CanonicalEventSizeError ? "content_too_large" - : "internal_error"; + : error instanceof DOMException && error.name === "AbortError" + ? "cancelled" + : "internal_error"; const code = normalizeDaemonRpcErrorCode(request.method, reportedCode); send({ kind: "error", @@ -963,13 +980,32 @@ export class AxlDaemon { controller?.abort(); return { cancellationRequested: controller !== undefined }; } + case "provider.list": + return this.providers().list(request.params, signal); + case "provider.catalog.refresh": + return this.providers().refresh(request.params, signal); + case "provider.auth.status": + return this.providers().authenticationStatus(request.params, signal); + case "provider.auth.login": + return this.providers().login(request.params, signal); + case "provider.auth.logout": + return this.providers().logout(request.params, signal); case "session.create": { - const { cwd, modelId, thinkingLevel, webFetch, webSearch, profile, requestSettings } = - request.params; + const { + cwd, + providerId, + modelId, + thinkingLevel, + webFetch, + webSearch, + profile, + requestSettings, + } = request.params; const reservation = this.creationReservation(acceptance); const created = await this.sessions.create( cwd, { + ...(providerId === undefined ? {} : { providerId }), ...(modelId === undefined ? {} : { modelId }), ...(requestSettings === undefined ? {} : { requestSettings }), ...(thinkingLevel === undefined ? {} : { thinkingLevel }), @@ -1074,11 +1110,20 @@ export class AxlDaemon { case "session.reload": return this.sessions.reload(request.params.sessionId, this.mutationOperationId(acceptance)); case "session.configure": { - const { sessionId, modelId, thinkingLevel, webFetch, webSearch, profile, requestSettings } = - request.params; + const { + sessionId, + providerId, + modelId, + thinkingLevel, + webFetch, + webSearch, + profile, + requestSettings, + } = request.params; return this.sessions.configure( sessionId, { + ...(providerId === undefined ? {} : { providerId }), ...(modelId === undefined ? {} : { modelId }), ...(requestSettings === undefined ? {} : { requestSettings }), ...(thinkingLevel === undefined ? {} : { thinkingLevel }), @@ -1152,6 +1197,16 @@ export class AxlDaemon { } } + private providers(): ProviderManagementService { + if (this.providerManagement === undefined) { + throw new DaemonError( + "unsupported_capability", + "Provider management is not available in this daemon", + ); + } + return this.providerManagement; + } + private mutationOperationId( acceptance: CommandAcceptance | undefined, ): ReturnType | undefined { diff --git a/packages/daemon/src/index.ts b/packages/daemon/src/index.ts index ab73d02..85d520f 100644 --- a/packages/daemon/src/index.ts +++ b/packages/daemon/src/index.ts @@ -4,5 +4,6 @@ export * from "./daemon.ts"; export * from "./event-migration.ts"; +export * from "./provider-management.ts"; export * from "./session-manager.ts"; export type { WireEvent } from "@axl/protocol"; diff --git a/packages/daemon/src/provider-management.ts b/packages/daemon/src/provider-management.ts new file mode 100644 index 0000000..d090247 --- /dev/null +++ b/packages/daemon/src/provider-management.ts @@ -0,0 +1,45 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import type { + ProviderAuthenticationStatusParams, + ProviderAuthenticationStatusResult, + ProviderCatalogRefreshParams, + ProviderCatalogRefreshResult, + ProviderListParams, + ProviderListResult, + ProviderLoginParams, + ProviderLoginResult, + ProviderLogoutParams, + ProviderLogoutResult, + ProviderRpcErrorCode, + ProviderRpcErrorDetails, +} from "@axl/protocol"; + +import { DaemonError } from "./session-manager.ts"; + +/** Daemon-owned provider operations implemented by the process runtime. */ +export interface ProviderManagementService { + list(params: ProviderListParams, signal?: AbortSignal): Promise; + refresh( + params: ProviderCatalogRefreshParams, + signal?: AbortSignal, + ): Promise; + authenticationStatus( + params: ProviderAuthenticationStatusParams, + signal?: AbortSignal, + ): Promise; + login(params: ProviderLoginParams, signal?: AbortSignal): Promise; + logout(params: ProviderLogoutParams, signal?: AbortSignal): Promise; + dispose?(): void | Promise; +} + +export class ProviderManagementError extends DaemonError { + declare readonly code: ProviderRpcErrorCode; + + constructor(code: ProviderRpcErrorCode, message: string, details: ProviderRpcErrorDetails) { + super(code, message, { details: { ...details } }); + this.name = "ProviderManagementError"; + this.code = code; + } +} diff --git a/packages/daemon/src/session-manager.ts b/packages/daemon/src/session-manager.ts index 94f2b69..3f235c6 100644 --- a/packages/daemon/src/session-manager.ts +++ b/packages/daemon/src/session-manager.ts @@ -95,6 +95,7 @@ export interface SessionRuntime { readonly compaction?: Partial; readonly retry?: ModelRetryOptions | false; readonly sandbox?: EventPayloadMap["sandbox.configured"]; + readonly configProvider?: EventPayloadMap["config.provider"]; readonly configModel?: EventPayloadMap["config.model"]; readonly configRequest?: EventPayloadMap["config.request"]; readonly configThinking?: EventPayloadMap["config.thinking"]; @@ -434,6 +435,7 @@ export class SessionManager { ...(runtime.compaction === undefined ? {} : { compaction: runtime.compaction }), ...(runtime.retry === undefined ? {} : { retry: runtime.retry }), ...(runtime.sandbox === undefined ? {} : { sandbox: runtime.sandbox }), + ...(runtime.configProvider === undefined ? {} : { configProvider: runtime.configProvider }), ...(runtime.configRequest === undefined ? {} : { configRequest: runtime.configRequest }), ...(runtime.configModel === undefined ? {} : { configModel: runtime.configModel }), ...(runtime.configThinking === undefined ? {} : { configThinking: runtime.configThinking }), @@ -484,6 +486,8 @@ export class SessionManager { events.length = 0; events.push(...stored.events); for (const event of events) this.authorizeEventBlobs(sessionId, event); + const configuredProvider = events.findLast((event) => event.type === "config.provider"); + const configuredModel = events.findLast((event) => event.type === "config.model"); const managed: ManagedSession = { session, cwd, @@ -491,7 +495,15 @@ export class SessionManager { listeners, activityListeners, activityState, - selection, + selection: { + ...selection, + ...(configuredProvider?.type === "config.provider" + ? { providerId: configuredProvider.payload.providerId } + : {}), + ...(configuredModel?.type === "config.model" + ? { modelId: configuredModel.payload.modelId } + : {}), + }, queuedInputs: Promise.resolve(), interactions: new Map(), queue: [], @@ -1063,6 +1075,7 @@ export class SessionManager { if (created?.type !== "session.created") { throw new DaemonError("corrupt_session", `Session ${sessionId} has no creation event`); } + let providerId: string | undefined; let modelId: string | undefined; let thinkingLevel: SessionConfiguration["thinkingLevel"]; let requestSettings: ModelRequestSettings | undefined; @@ -1070,7 +1083,8 @@ export class SessionManager { let webSearch: boolean | undefined; let profile: SessionConfiguration["profile"]; for (const event of events) { - if (event.type === "config.model") modelId = event.payload.modelId; + if (event.type === "config.provider") providerId = event.payload.providerId; + else if (event.type === "config.model") modelId = event.payload.modelId; else if (event.type === "config.request") requestSettings = event.payload; else if (event.type === "config.thinking") thinkingLevel = event.payload.requested; else if (event.type === "config.profile") profile = event.payload.profile; @@ -1080,6 +1094,9 @@ export class SessionManager { } } return this.open(sessionId, created.payload.cwd, { + // Event-format v1 sessions created before provider selection was logged + // always used Azure OpenAI Responses. + providerId: providerId ?? "azure-openai-responses", ...(requestSettings === undefined ? {} : { requestSettings }), ...(modelId === undefined ? {} : { modelId }), ...(thinkingLevel === undefined ? {} : { thinkingLevel }), @@ -1113,6 +1130,7 @@ export class SessionManager { update: SessionConfiguration, operationId?: OperationId, ): Promise<{ + providerId: string; modelId: string; requestedThinkingLevel: NonNullable; effectiveThinkingLevel: NonNullable; @@ -1158,7 +1176,8 @@ export class SessionManager { ); } const boundary: SessionRuntimeBoundary = - update.modelId !== undefined && update.modelId !== managed.selection.modelId + (update.providerId !== undefined && update.providerId !== managed.selection.providerId) || + (update.modelId !== undefined && update.modelId !== managed.selection.modelId) ? "model_switch" : (update.webFetch !== undefined && update.webFetch !== managed.selection.webFetch) || (update.webSearch !== undefined && update.webSearch !== managed.selection.webSearch) @@ -1166,14 +1185,26 @@ export class SessionManager { : "config_change"; const before = managed.events.length; await this.rebuild(managed, boundary, selection, operationId); - managed.selection = selection; - return this.configurationResult(managed, managed.events.slice(before)); + const boundaryEvents = managed.events.slice(before); + const configuredProvider = boundaryEvents.findLast((event) => event.type === "config.provider"); + const configuredModel = boundaryEvents.findLast((event) => event.type === "config.model"); + managed.selection = { + ...selection, + ...(configuredProvider?.type === "config.provider" + ? { providerId: configuredProvider.payload.providerId } + : {}), + ...(configuredModel?.type === "config.model" + ? { modelId: configuredModel.payload.modelId } + : {}), + }; + return this.configurationResult(managed, boundaryEvents); } private configurationResult( managed: ManagedSession, boundaryEvents: readonly CanonicalEvent[], ): { + providerId: string; modelId: string; requestedThinkingLevel: NonNullable; effectiveThinkingLevel: NonNullable; @@ -1183,12 +1214,13 @@ export class SessionManager { requestSettings: ModelRequestSettings; boundaryEventIds: readonly EventId[]; } { - const model = managed.events.findLast((event) => event.type === "config.model"); - const modelId = - managed.selection.modelId ?? - (model?.type === "config.model" ? model.payload.modelId : undefined); - if (modelId === undefined) { - throw new DaemonError("corrupt_session", "Configured session has no model identity"); + const providerId = managed.selection.providerId; + const modelId = managed.selection.modelId; + if (providerId === undefined || modelId === undefined) { + throw new DaemonError( + "corrupt_session", + "Configured session has no provider and model identity", + ); } const thinking = managed.events.findLast((event) => event.type === "config.thinking"); const tools = managed.events.findLast((event) => event.type === "config.tools"); @@ -1197,6 +1229,7 @@ export class SessionManager { (thinking?.type === "config.thinking" ? thinking.payload.requested : "off"); const request = managed.events.findLast((event) => event.type === "config.request"); return { + providerId, modelId, requestedThinkingLevel, requestSettings: @@ -1375,7 +1408,11 @@ export class SessionManager { const active = deferredTurn("turn", operationId); managed.activeTurn = active; try { - await this.captureWorkspaceCheckpoint(managed); + try { + await this.captureWorkspaceCheckpoint(managed, active.controller.signal); + } catch (error) { + if (!active.controller.signal.aborted) throw error; + } let result = await managed.session.runTurn( content, active.controller.signal, @@ -1533,7 +1570,11 @@ export class SessionManager { const active = deferredTurn("shell", operationId); managed.activeTurn = active; try { - await this.captureWorkspaceCheckpoint(managed); + try { + await this.captureWorkspaceCheckpoint(managed, active.controller.signal); + } catch (error) { + if (!active.controller.signal.aborted) throw error; + } const event = await managed.session.runShell( command, excluded, @@ -1682,10 +1723,13 @@ export class SessionManager { } } - private async captureWorkspaceCheckpoint(managed: ManagedSession): Promise { + private async captureWorkspaceCheckpoint( + managed: ManagedSession, + signal?: AbortSignal, + ): Promise { if (!managed.workspaceCheckpointsEnabled) return; try { - await this.workspaceCheckpoints.capture(managed.session.log.sessionId, managed.cwd); + await this.workspaceCheckpoints.capture(managed.session.log.sessionId, managed.cwd, signal); delete managed.checkpointError; } catch (error) { if (error instanceof WorkspaceCheckpointError) { diff --git a/packages/daemon/src/workspace-checkpoint.ts b/packages/daemon/src/workspace-checkpoint.ts index 3260c32..a7dc322 100644 --- a/packages/daemon/src/workspace-checkpoint.ts +++ b/packages/daemon/src/workspace-checkpoint.ts @@ -49,25 +49,31 @@ export class WorkspaceCheckpointStore { } } - capture(sessionId: SessionId, cwd: string): Promise { - return this.serialized(sessionId, () => this.captureUnlocked(sessionId, cwd)); + capture(sessionId: SessionId, cwd: string, signal?: AbortSignal): Promise { + return this.serialized(sessionId, () => this.captureUnlocked(sessionId, cwd, signal)); } - private async captureUnlocked(sessionId: SessionId, cwd: string): Promise { + private async captureUnlocked( + sessionId: SessionId, + cwd: string, + signal?: AbortSignal, + ): Promise { const paths = this.paths(sessionId); - await this.assertGitWorkspace(cwd); - await this.assertBoundedWorkspace(cwd); + await this.assertGitWorkspace(cwd, signal); + await this.assertBoundedWorkspace(cwd, signal); await mkdir(paths.root, { recursive: true, mode: 0o700 }); try { await stat(paths.git); } catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; - await this.git(cwd, ["init", "--quiet", "--bare", paths.git]); - await this.snapshotGit(cwd, paths.git, ["config", "core.autocrlf", "false"]); - await this.snapshotGit(cwd, paths.git, ["config", "core.hooksPath", "/dev/null"]); + await this.git(cwd, ["init", "--quiet", "--bare", paths.git], signal); + await this.snapshotGit(cwd, paths.git, ["config", "core.autocrlf", "false"], signal); + await this.snapshotGit(cwd, paths.git, ["config", "core.hooksPath", "/dev/null"], signal); } - await this.snapshotGit(cwd, paths.git, ["add", "-A", "--", "."]); - const tree = decodeGit((await this.snapshotGit(cwd, paths.git, ["write-tree"])).stdout).trim(); + await this.snapshotGit(cwd, paths.git, ["add", "-A", "--", "."], signal); + const tree = decodeGit( + (await this.snapshotGit(cwd, paths.git, ["write-tree"], signal)).stdout, + ).trim(); const record: CheckpointRecord = { version: 1, checkpointId: randomUUID(), tree }; const temporary = `${paths.record}.${randomUUID()}.tmp`; try { diff --git a/packages/daemon/test/daemon.test.ts b/packages/daemon/test/daemon.test.ts index d5911ea..c45cd24 100644 --- a/packages/daemon/test/daemon.test.ts +++ b/packages/daemon/test/daemon.test.ts @@ -27,7 +27,7 @@ import { StringDecoder } from "node:string_decoder"; import test, { type TestContext } from "node:test"; import { promisify } from "node:util"; -import { type ModelPort, type ModelRetryOptions, ToolRegistry } from "@axl/kernel"; +import { JsonlEventLog, type ModelPort, type ModelRetryOptions, ToolRegistry } from "@axl/kernel"; import type { CanonicalEvent, ModelStreamEvent, @@ -3484,23 +3484,78 @@ test("ignores incomplete migration targets until their manifest is published", a }); }); +test("provider-less version-1 sessions resume with the legacy Azure provider", async (context) => { + const directory = await mkdtemp(join(tmpdir(), "axl-daemon-legacy-provider-")); + context.after(() => rm(directory, { recursive: true, force: true })); + const dataDirectory = join(directory, "data"); + const sessionId = parseSessionId("123e4567-e89b-42d3-a456-426614170111"); + const { log } = await JsonlEventLog.open( + join(dataDirectory, "sessions", `${sessionId}.jsonl`), + sessionId, + ); + await log.append({ + version: 1, + id: "00000000-0000-4000-8000-000000000001", + sessionId, + parentId: null, + timestamp: 1, + type: "session.created", + payload: { cwd: directory }, + }); + await log.append({ + version: 1, + id: "00000000-0000-4000-8000-000000000002", + sessionId, + parentId: "00000000-0000-4000-8000-000000000001", + timestamp: 2, + type: "config.model", + payload: { modelId: "gpt-5" }, + }); + const selections: Array<{ providerId?: string; modelId?: string }> = []; + const socketPath = join(directory, "axl.sock"); + const daemon = new AxlDaemon({ + socketPath, + dataDirectory, + runtime: ({ selection }) => { + selections.push(selection); + return { model: replyPort(), tools: new ToolRegistry() }; + }, + }); + await daemon.start(); + context.after(() => daemon.stop()); + const client = await connectUnixClient(socketPath); + context.after(() => client.close()); + await client.request("session.resume", { sessionId }); + assert.equal(selections[0]?.providerId, "azure-openai-responses"); + assert.equal(selections[0]?.modelId, "gpt-5"); +}); + test("configuration changes rebuild and log the selected model and thinking", async (context) => { const directory = await mkdtemp(join(tmpdir(), "axl-daemon-")); context.after(() => rm(directory, { recursive: true, force: true })); const socketPath = join(directory, "axl.sock"); - const configured: Array<{ boundary: string; model?: string; thinking?: string }> = []; + const configured: Array<{ + boundary: string; + provider?: string; + model?: string; + thinking?: string; + }> = []; const daemon = new AxlDaemon({ socketPath, dataDirectory: join(directory, "data"), runtime: ({ boundary, selection }) => { configured.push({ boundary, + ...(selection.providerId === undefined ? {} : { provider: selection.providerId }), ...(selection.modelId === undefined ? {} : { model: selection.modelId }), ...(selection.thinkingLevel === undefined ? {} : { thinking: selection.thinkingLevel }), }); return { model: replyPort(), tools: new ToolRegistry(), + ...(selection.providerId === undefined + ? {} + : { configProvider: { providerId: selection.providerId } }), configRequest: selection.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS, ...(selection.modelId === undefined ? {} : { configModel: { modelId: selection.modelId } }), ...(selection.thinkingLevel === undefined @@ -3522,6 +3577,7 @@ test("configuration changes rebuild and log the selected model and thinking", as context.after(() => client.close()); const created = await client.request("session.create", { cwd: directory, + providerId: "openai", modelId: "gpt-5", thinkingLevel: "medium", requestSettings: { maxOutputTokens: null, httpIdleTimeoutMs: 300_000 }, @@ -3531,6 +3587,7 @@ test("configuration changes rebuild and log the selected model and thinking", as "session.configure", { sessionId: created.sessionId, + providerId: "openai", modelId: "gpt-4.1", thinkingLevel: "high", requestSettings: { maxOutputTokens: 2048, httpIdleTimeoutMs: 0 }, @@ -3539,9 +3596,10 @@ test("configuration changes rebuild and log the selected model and thinking", as ); assert.deepEqual(configured, [ - { boundary: "session_start", model: "gpt-5", thinking: "medium" }, - { boundary: "model_switch", model: "gpt-4.1", thinking: "high" }, + { boundary: "session_start", provider: "openai", model: "gpt-5", thinking: "medium" }, + { boundary: "model_switch", provider: "openai", model: "gpt-4.1", thinking: "high" }, ]); + assert.equal(changed.providerId, "openai"); assert.equal(changed.modelId, "gpt-4.1"); assert.equal(changed.requestedThinkingLevel, "high"); assert.equal(changed.effectiveThinkingLevel, "high"); @@ -3549,7 +3607,7 @@ test("configuration changes rebuild and log the selected model and thinking", as assert.equal(changed.profile, "standard"); assert.equal(changed.webFetch, false); assert.equal(changed.webSearch, false); - assert.equal(changed.boundaryEventIds.length, 3); + assert.equal(changed.boundaryEventIds.length, 4); client.close(); await daemon.stop(); @@ -3560,6 +3618,9 @@ test("configuration changes rebuild and log the selected model and thinking", as runtime: ({ selection }) => ({ model: replyPort(), tools: new ToolRegistry(), + ...(selection.providerId === undefined + ? {} + : { configProvider: { providerId: selection.providerId } }), configRequest: selection.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS, ...(selection.modelId === undefined ? {} : { configModel: { modelId: selection.modelId } }), ...(selection.thinkingLevel === undefined @@ -3582,6 +3643,7 @@ test("configuration changes rebuild and log the selected model and thinking", as "session.configure", { sessionId: created.sessionId, + providerId: "openai", modelId: "gpt-4.1", thinkingLevel: "high", requestSettings: { maxOutputTokens: 2048, httpIdleTimeoutMs: 0 }, diff --git a/packages/daemon/test/provider-management.test.ts b/packages/daemon/test/provider-management.test.ts new file mode 100644 index 0000000..982c2a3 --- /dev/null +++ b/packages/daemon/test/provider-management.test.ts @@ -0,0 +1,230 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import { mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test, { type TestContext } from "node:test"; + +import type { ModelPort } from "@axl/kernel"; +import { ToolRegistry } from "@axl/kernel"; +import type { ProviderManagementService } from "../src/provider-management.ts"; +import { AxlClientError, ProviderClientError } from "@axl/sdk"; +import { connectUnixClient } from "@axl/sdk/unix"; + +import { AxlDaemon, ProviderManagementError } from "../src/index.ts"; + +const model: ModelPort = { + stream: () => + (async function* () { + yield { + type: "completed" as const, + stopReason: "stop" as const, + usage: { inputTokens: 0, outputTokens: 0, cacheReadTokens: 0, cacheWriteTokens: 0 }, + }; + })(), +}; + +function inventory() { + return { + providers: [ + { + providerId: "openrouter", + displayName: "OpenRouter", + enabled: true, + authMethods: ["environment", "file", "oauth"] as const, + loginMethods: ["api_key", "oauth"] as const, + authentication: { providerId: "openrouter", phase: "idle" as const }, + catalog: { refreshable: true }, + models: [ + { + providerId: "openrouter", + modelId: "example/model", + displayName: "Example Model", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: false, imageInput: true }, + reasoning: true, + supportedThinkingLevels: ["off", "low", "medium", "high"] as const, + contextWindow: 128_000, + maxOutputTokens: 16_384, + availability: { status: "available" as const }, + }, + ], + }, + ], + }; +} + +function service(overrides: Partial = {}): ProviderManagementService { + return { + list: async () => inventory(), + refresh: async () => ({ + providers: [{ providerId: "openrouter", status: "refreshed", modelCount: 1 }], + }), + authenticationStatus: async () => ({ + providers: [{ providerId: "openrouter", phase: "idle" }], + }), + login: async ({ providerId, method }) => ({ + providerId, + method, + phase: "authenticated", + source: "test adapter", + }), + logout: async ({ providerId }) => ({ providerId, phase: "logged_out" }), + ...overrides, + }; +} + +async function start( + context: TestContext, + providerManagement?: ProviderManagementService, +): Promise<{ daemon: AxlDaemon; socketPath: string }> { + const directory = await mkdtemp(join(tmpdir(), "axl-provider-management-")); + context.after(() => rm(directory, { recursive: true, force: true })); + const socketPath = join(directory, "axl.sock"); + const daemon = new AxlDaemon({ + socketPath, + dataDirectory: join(directory, "data"), + ...(providerManagement === undefined ? {} : { providerManagement }), + runtime: async () => ({ model, tools: new ToolRegistry() }), + }); + await daemon.start(); + context.after(() => daemon.stop()); + return { daemon, socketPath }; +} + +test("grants provider capabilities only when the daemon owns a service", async (context) => { + const without = await start(context); + const unsupported = await connectUnixClient(without.socketPath); + context.after(() => unsupported.close()); + assert.equal(unsupported.connection.grantedCapabilities.includes("provider.list"), false); + await assert.rejects( + unsupported.listProviders(), + (error) => error instanceof AxlClientError && error.code === "unsupported_capability", + ); + + const withService = await start(context, service()); + const client = await connectUnixClient(withService.socketPath); + context.after(() => client.close()); + assert.equal(client.connection.grantedCapabilities.includes("provider.list"), true); + assert.deepEqual(await client.listProviders(), inventory()); +}); + +test("serves typed authentication actions without credential payloads", async (context) => { + const started = await start(context, service()); + const client = await connectUnixClient(started.socketPath); + context.after(() => client.close()); + + assert.deepEqual(await client.providerAuthenticationStatus({ providerId: "openrouter" }), { + providers: [{ providerId: "openrouter", phase: "idle" }], + }); + assert.deepEqual(await client.loginProvider({ providerId: "openrouter", method: "oauth" }), { + providerId: "openrouter", + method: "oauth", + phase: "authenticated", + source: "test adapter", + }); + assert.deepEqual(await client.logoutProvider({ providerId: "openrouter" }), { + providerId: "openrouter", + phase: "logged_out", + }); +}); + +test("cancels explicit catalog refresh", async (context) => { + let began!: () => void; + const beginning = new Promise((resolve) => { + began = resolve; + }); + const started = await start( + context, + service({ + refresh: async (_params, signal) => { + began(); + await new Promise((_resolve, reject) => { + signal?.addEventListener("abort", () => reject(signal.reason), { once: true }); + }); + return { providers: [] }; + }, + }), + ); + const client = await connectUnixClient(started.socketPath); + context.after(() => client.close()); + const controller = new AbortController(); + const refresh = client.refreshProviderCatalogs({}, { signal: controller.signal }); + await beginning; + controller.abort(); + await assert.rejects( + refresh, + (error) => error instanceof AxlClientError && error.code === "cancelled", + ); +}); + +test("cancels trusted-host provider login", async (context) => { + let began!: () => void; + const beginning = new Promise((resolve) => { + began = resolve; + }); + const started = await start( + context, + service({ + login: async (_params, signal) => { + began(); + await new Promise((_resolve, reject) => { + signal?.addEventListener("abort", () => reject(signal.reason), { once: true }); + }); + return { providerId: "openrouter", phase: "authenticated" }; + }, + }), + ); + const client = await connectUnixClient(started.socketPath); + context.after(() => client.close()); + const controller = new AbortController(); + const login = client.loginProvider( + { providerId: "openrouter", method: "oauth" }, + { signal: controller.signal }, + ); + await beginning; + controller.abort(); + await assert.rejects( + login, + (error) => error instanceof AxlClientError && error.code === "cancelled", + ); +}); + +test("returns actionable provider failures and redacts unknown causes", async (context) => { + const secret = "provider-secret-value"; + const actionable = await start( + context, + service({ + login: async ({ providerId }) => { + throw new ProviderManagementError( + "authentication_required", + `Provider ${providerId} requires authentication`, + { category: "authentication", action: "login", providerId }, + ); + }, + list: async () => { + throw new Error(secret); + }, + }), + ); + const client = await connectUnixClient(actionable.socketPath); + context.after(() => client.close()); + + await assert.rejects( + client.loginProvider({ providerId: "openrouter", method: "oauth" }), + (error) => + error instanceof ProviderClientError && + error.code === "authentication_required" && + error.details?.category === "authentication" && + error.details.action === "login", + ); + await assert.rejects( + client.listProviders(), + (error) => + error instanceof AxlClientError && + error.code === "internal_error" && + !error.message.includes(secret), + ); +}); diff --git a/packages/kernel/src/agent-session.ts b/packages/kernel/src/agent-session.ts index 543d947..69487b5 100644 --- a/packages/kernel/src/agent-session.ts +++ b/packages/kernel/src/agent-session.ts @@ -168,6 +168,8 @@ export interface AgentSessionOptions { readonly log?: EventLogOptions; /** Sandbox state announced at every open as a `sandbox.configured` event. */ readonly sandbox?: EventPayloadMap["sandbox.configured"]; + /** Provider configuration announced at every open as a `config.provider` event. */ + readonly configProvider?: EventPayloadMap["config.provider"]; /** Model configuration announced at every open as a `config.model` event. */ readonly configModel?: EventPayloadMap["config.model"]; readonly configRequest?: EventPayloadMap["config.request"]; @@ -334,6 +336,9 @@ export class AgentSession { if (options.sandbox !== undefined) { await session.append(options.boundaryOperationId, "sandbox.configured", options.sandbox); } + if (options.configProvider !== undefined) { + await session.append(options.boundaryOperationId, "config.provider", options.configProvider); + } if (options.configRequest !== undefined) await session.append(options.boundaryOperationId, "config.request", options.configRequest); if (options.configModel !== undefined) { diff --git a/packages/kernel/src/jsonl-event-log.ts b/packages/kernel/src/jsonl-event-log.ts index dc3f060..dfa8f66 100644 --- a/packages/kernel/src/jsonl-event-log.ts +++ b/packages/kernel/src/jsonl-event-log.ts @@ -19,7 +19,8 @@ import { import { redactEventForStorage } from "./redaction.ts"; export interface EventLogOptions { - readonly secretValues?: readonly string[]; + /** Read at each append so rotating credentials are redacted before persistence. */ + readonly secretValues?: readonly string[] | (() => readonly string[]); /** Daemon-owned transformation available only when canonical encoding exceeds the limit. */ readonly prepareOversizedEvent?: ( event: CanonicalEvent, @@ -223,7 +224,7 @@ async function appendDurably(path: string, line: Uint8Array): Promise { export class JsonlEventLog { readonly path: string; readonly sessionId: SessionId; - private readonly secretValues: readonly string[]; + private readonly secretValues: () => readonly string[]; private readonly prepareOversizedEvent: | ((event: CanonicalEvent) => CanonicalEvent | Promise) | undefined; @@ -232,7 +233,9 @@ export class JsonlEventLog { private constructor(path: string, sessionId: SessionId, options: EventLogOptions) { this.path = path; this.sessionId = sessionId; - this.secretValues = [...(options.secretValues ?? [])]; + const secretValues = options.secretValues; + this.secretValues = + typeof secretValues === "function" ? secretValues : () => secretValues ?? []; this.prepareOversizedEvent = options.prepareOversizedEvent; } @@ -252,7 +255,7 @@ export class JsonlEventLog { append(value: unknown): Promise { let redacted: CanonicalEvent; try { - redacted = redactEventForStorage(value, this.secretValues); + redacted = redactEventForStorage(value, this.secretValues()); if (redacted.sessionId !== this.sessionId) { throw new ProtocolValidationError( "event.sessionId", diff --git a/packages/kernel/test/jsonl-event-log.test.ts b/packages/kernel/test/jsonl-event-log.test.ts index 8630e40..9bfe4e4 100644 --- a/packages/kernel/test/jsonl-event-log.test.ts +++ b/packages/kernel/test/jsonl-event-log.test.ts @@ -132,6 +132,31 @@ test("serializes appends and redacts structured secret fields before writing", a } }); +test("reads rotating redaction values at every append", async (context) => { + const secrets = new Set(["first-provider-secret"]); + const { path, log } = await openTemporaryLog(context, { + secretValues: () => [...secrets], + }); + await log.append( + makeEvent(1, "assistant.message", { + content: [{ type: "text", text: "echo first-provider-secret" }], + stopReason: "stop", + }), + ); + secrets.add("rotated-provider-secret"); + await log.append( + makeEvent(2, "tool.call", { + callId: "call-1", + name: "echo", + input: { value: "rotated-provider-secret" }, + }), + ); + const raw = await readFile(path, "utf8"); + assert.equal(raw.includes("first-provider-secret"), false); + assert.equal(raw.includes("rotated-provider-secret"), false); + assert.equal(raw.includes(REDACTED_VALUE), true); +}); + test("discards only a torn final line before accepting another append", async (context) => { const { path, log } = await openTemporaryLog(context); await log.append(makeEvent(1, "session.created", { cwd: "/workspace" })); diff --git a/packages/protocol/scripts/generate-conformance.ts b/packages/protocol/scripts/generate-conformance.ts index 18ab6c4..9cf8086 100644 --- a/packages/protocol/scripts/generate-conformance.ts +++ b/packages/protocol/scripts/generate-conformance.ts @@ -12,6 +12,7 @@ import { EVENT_TYPES, type EventPayloadMap, type EventType, + isProviderRpcErrorCode, isRetryableMutationMethod, isRpcErrorAllowed, isRpcErrorRetryable, @@ -170,7 +171,17 @@ const params = { }, "connection.ping": {}, "request.cancel": { requestId: 7 }, - "session.create": { cwd: "/workspace", profile: "standard" }, + "provider.list": {}, + "provider.catalog.refresh": { providerId: "provider-1" }, + "provider.auth.status": { providerId: "provider-1" }, + "provider.auth.login": { providerId: "provider-1", method: "oauth" }, + "provider.auth.logout": { providerId: "provider-1" }, + "session.create": { + cwd: "/workspace", + providerId: "provider-1", + modelId: "model-1", + profile: "standard", + }, "session.resume": { sessionId }, "session.list": { scope: "all_local", order: "recent", pageSize: 50 }, "session.history": { snapshotId: "snapshot-1", pageCursor: "page-1" }, @@ -195,6 +206,7 @@ const params = { "session.reload": { sessionId }, "session.configure": { sessionId, + providerId: "provider-1", modelId: "model-1", thinkingLevel: "medium", requestSettings: { maxOutputTokens: null, httpIdleTimeoutMs: 300_000 }, @@ -252,6 +264,62 @@ const results = { }, "connection.ping": {}, "request.cancel": { cancellationRequested: true }, + "provider.list": { + providers: [ + { + providerId: "provider-1", + displayName: "Provider One", + enabled: true, + regionFamily: "provider", + region: "global", + authMethods: ["environment", "oauth"], + loginMethods: ["api_key", "oauth"], + authentication: { + providerId: "provider-1", + phase: "authenticated", + method: "oauth", + source: "OAuth", + }, + catalog: { + refreshable: true, + generation: 1, + checkedAt: 1, + updatedAt: 1, + source: { id: "provider.catalog", kind: "provider_api", revision: "v1" }, + }, + models: [ + { + providerId: "provider-1", + modelId: "model-1", + displayName: "Model One", + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: false }, + reasoning: true, + supportedThinkingLevels: ["off", "low", "medium", "high"], + contextWindow: 128000, + maxOutputTokens: 16384, + cost: { inputUsdPerMTok: 1, outputUsdPerMTok: 2 }, + availability: { status: "available" }, + }, + ], + }, + ], + }, + "provider.catalog.refresh": { + providers: [{ providerId: "provider-1", status: "refreshed", modelCount: 1 }], + }, + "provider.auth.status": { + providers: [ + { providerId: "provider-1", phase: "authenticated", method: "oauth", source: "OAuth" }, + ], + }, + "provider.auth.login": { + providerId: "provider-1", + phase: "authenticated", + method: "oauth", + source: "OAuth", + }, + "provider.auth.logout": { providerId: "provider-1", phase: "logged_out" }, "session.create": opened, "session.resume": opened, "session.list": { @@ -294,6 +362,7 @@ const results = { "session.interrupt": { interrupted: true, operationId }, "session.reload": { boundaryEventIds: [eventId] }, "session.configure": { + providerId: "provider-1", modelId: "model-1", requestedThinkingLevel: "medium", effectiveThinkingLevel: "medium", @@ -385,7 +454,14 @@ const errors = RPC_ERROR_CODES.map((code, index) => { return { kind: "error" as const, id: -1, - error: { code, message: `Fixture error: ${code}`, retryable: isRpcErrorRetryable(code) }, + error: { + code, + message: `Fixture error: ${code}`, + retryable: isRpcErrorRetryable(code), + ...(isProviderRpcErrorCode(code) + ? { details: { category: "provider", action: "configure_provider" } } + : {}), + }, }; } const method = RPC_METHODS.find((candidate) => isRpcErrorAllowed(candidate, code)); @@ -394,7 +470,14 @@ const errors = RPC_ERROR_CODES.map((code, index) => { kind: "error" as const, id: index + 1, method, - error: { code, message: `Fixture error: ${code}`, retryable: isRpcErrorRetryable(code) }, + error: { + code, + message: `Fixture error: ${code}`, + retryable: isRpcErrorRetryable(code), + ...(isProviderRpcErrorCode(code) + ? { details: { category: "provider", action: "configure_provider" } } + : {}), + }, }; }); const allowedErrors = RPC_METHODS.flatMap((method, methodIndex) => @@ -407,6 +490,9 @@ const allowedErrors = RPC_METHODS.flatMap((method, methodIndex) => code, message: `Fixture ${method} error: ${code}`, retryable: isRpcErrorRetryable(code), + ...(isProviderRpcErrorCode(code) + ? { details: { category: "provider", action: "configure_provider" } } + : {}), }, }), ), @@ -483,10 +569,14 @@ const document = { serverMessages, events, }; -const output = `${JSON.stringify(document, null, 2).replace( - /\[\n\s+("(?:[^"\\]|\\.)*")\n\s*\]/g, - "[$1]", -)}\n`; +const output = `${JSON.stringify(document, null, 2) + .replace(/\[\n\s+("(?:[^"\\]|\\.)*")\n\s*\]/g, "[$1]") + .replace(/\[\n\s+"environment",\n\s+"oauth"\n\s*\]/g, '["environment", "oauth"]') + .replace(/\[\n\s+"api_key",\n\s+"oauth"\n\s*\]/g, '["api_key", "oauth"]') + .replace( + /\[\n\s+"off",\n\s+"low",\n\s+"medium",\n\s+"high"\n\s*\]/g, + '["off", "low", "medium", "high"]', + )}\n`; const defaultTarget = resolve( dirname(fileURLToPath(import.meta.url)), "../test/fixtures/conformance.json", diff --git a/packages/protocol/src/index.ts b/packages/protocol/src/index.ts index 204e991..60892b7 100644 --- a/packages/protocol/src/index.ts +++ b/packages/protocol/src/index.ts @@ -7,6 +7,7 @@ export * from "./canonical-request.ts"; export * from "./event-envelope.ts"; export * from "./events.ts"; export * from "./model-stream.ts"; +export * from "./provider-management.ts"; export * from "./version.ts"; export * from "./wire.ts"; export * from "./host-control.ts"; diff --git a/packages/protocol/src/model-stream.ts b/packages/protocol/src/model-stream.ts index 112968c..e0aff81 100644 --- a/packages/protocol/src/model-stream.ts +++ b/packages/protocol/src/model-stream.ts @@ -1,4 +1,5 @@ // SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-FileCopyrightText: 2026 Kaushik Kumar // SPDX-License-Identifier: Apache-2.0 import type { JsonObject } from "./event-envelope.ts"; @@ -78,28 +79,105 @@ export type ModelErrorCategory = export type ModelRequestPhase = "before_dispatch" | "awaiting_response" | "streaming" | "unknown"; +export interface ProviderRetryGuidance { + readonly retryAfterMs?: number; + readonly resetAtEpochMs?: number; +} + +/** A deliberately bounded diagnostic shape that cannot carry headers or arbitrary provider data. */ +export interface SafeProviderDiagnostic { + readonly code: string; + readonly message: string; + readonly severity: "info" | "warning" | "error"; +} + +/** Provider attribution safe to persist with a canonical response. */ +export interface ProviderResponseMetadata { + readonly providerId: string; + readonly requestedModelId: string; + readonly routedModelId?: string; + readonly responseId?: string; + readonly nativeStopReason?: string; + readonly latencyMs?: number; +} + export interface ModelStreamError { readonly code: string; readonly message: string; /** True only when re-dispatching the identical request is known to be safe. */ readonly retryable: boolean; + /** True when content was emitted before this failure. */ + readonly partial?: boolean; readonly category?: ModelErrorCategory; readonly requestPhase?: ModelRequestPhase; + /** Compatibility field consumed by the kernel retry coordinator. */ readonly retryAfterMs?: number; + readonly retry?: ProviderRetryGuidance; + readonly response?: ProviderResponseMetadata; + readonly diagnostics?: readonly SafeProviderDiagnostic[]; +} + +interface PositionedContent { + /** Stable provider-neutral position for interleaved response blocks. */ + readonly contentIndex?: number; +} + +/** + * Opaque provider replay data attached to one completed response block. + * Values are provenance-bound and must never contain authentication material. + */ +export interface ProviderReplayMetadata { + readonly target: "thinking" | "text" | "tool_call"; + readonly contentIndex: number; + readonly providerId: string; + readonly apiDialect: string; + readonly modelId: string; + readonly callId?: string; + readonly signature?: string; + /** True when the signature is an opaque redacted-thinking payload. */ + readonly redacted?: boolean; + readonly responseId?: string; + readonly itemId?: string; + readonly namespace?: string; } /** * Canonical model stream shape. Every stream yields zero or more deltas and * tool calls, then exactly one terminal event: `completed`, `error`, or * `aborted`. Nothing follows a terminal event. + * + * Existing adapters may omit `contentIndex`. Adapters that can interleave + * blocks provide it. A complete `tool_call` remains authoritative, while the + * start and delta events are optional progress for clients. */ export type ModelStreamEvent = - | { readonly type: "text_delta"; readonly text: string } - | { readonly type: "thinking_delta"; readonly text: string } - | ({ readonly type: "tool_call" } & ToolCallRequest) - | { readonly type: "completed"; readonly stopReason: AssistantStopReason; readonly usage: Usage } + | ({ readonly type: "text_delta"; readonly text: string } & PositionedContent) + | ({ readonly type: "thinking_delta"; readonly text: string } & PositionedContent) + | ({ + readonly type: "tool_call_start"; + readonly contentIndex: number; + readonly callId: string; + readonly name: string; + } & PositionedContent) + | ({ + readonly type: "tool_call_delta"; + readonly contentIndex: number; + readonly callId: string; + readonly argumentsDelta: string; + } & PositionedContent) + | ({ readonly type: "tool_call" } & ToolCallRequest & PositionedContent) + | ({ readonly type: "replay_metadata" } & ProviderReplayMetadata) + | { + readonly type: "completed"; + readonly stopReason: AssistantStopReason; + readonly usage: Usage; + /** True when the provider intentionally returned usable but incomplete content. */ + readonly partial?: boolean; + readonly response?: ProviderResponseMetadata; + readonly diagnostics?: readonly SafeProviderDiagnostic[]; + } | ({ readonly type: "error" } & ModelStreamError) - | { readonly type: "aborted" }; + | { readonly type: "aborted"; readonly partial?: boolean }; export type TerminalModelStreamEvent = Extract< ModelStreamEvent, @@ -111,3 +189,289 @@ export function isTerminalModelStreamEvent( ): event is TerminalModelStreamEvent { return event.type === "completed" || event.type === "error" || event.type === "aborted"; } + +export class ModelStreamValidationError extends Error { + readonly path: string; + + constructor(path: string, message: string) { + super(`${path} ${message}`); + this.name = "ModelStreamValidationError"; + this.path = path; + } +} + +function fail(path: string, message: string): never { + throw new ModelStreamValidationError(path, message); +} + +function object(value: unknown, path: string): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + fail(path, "must be an object"); + } + return value as Record; +} + +function exact( + value: Record, + path: string, + required: readonly string[], + optional: readonly string[] = [], +): void { + const allowed = new Set([...required, ...optional]); + for (const key of Object.keys(value)) { + if (!allowed.has(key)) fail(`${path}.${key}`, "is not allowed"); + } + for (const key of required) { + if (!Object.hasOwn(value, key)) fail(`${path}.${key}`, "is required"); + } +} + +function string(value: unknown, path: string, allowEmpty = false): string { + if (typeof value !== "string" || (!allowEmpty && value.length === 0)) { + fail(path, allowEmpty ? "must be a string" : "must be a non-empty string"); + } + return value; +} + +function boolean(value: unknown, path: string): boolean { + if (typeof value !== "boolean") fail(path, "must be a boolean"); + return value; +} + +function nonNegativeNumber(value: unknown, path: string, integer = false): number { + if ( + typeof value !== "number" || + !Number.isFinite(value) || + value < 0 || + (integer && !Number.isSafeInteger(value)) + ) { + fail(path, integer ? "must be a non-negative safe integer" : "must be a non-negative number"); + } + return value; +} + +function optionalPosition(value: Record, path: string): void { + if (value.contentIndex !== undefined) { + nonNegativeNumber(value.contentIndex, `${path}.contentIndex`, true); + } +} + +function validateJson(value: unknown, path: string, ancestors = new Set()): void { + if ( + value === null || + typeof value === "string" || + typeof value === "boolean" || + (typeof value === "number" && Number.isFinite(value)) + ) { + return; + } + if (typeof value !== "object") fail(path, "must be JSON-compatible"); + if (ancestors.has(value)) fail(path, "must not contain cycles"); + const next = new Set(ancestors); + next.add(value); + if (Array.isArray(value)) { + value.forEach((item, index) => { + validateJson(item, `${path}[${index}]`, next); + }); + return; + } + const prototype = Object.getPrototypeOf(value); + if (prototype !== Object.prototype && prototype !== null) fail(path, "must be a plain object"); + for (const [key, item] of Object.entries(value)) validateJson(item, `${path}.${key}`, next); +} + +function validateUsage(value: unknown, path: string): void { + const usage = object(value, path); + exact( + usage, + path, + ["inputTokens", "outputTokens", "cacheReadTokens", "cacheWriteTokens"], + ["reasoningTokens", "costUsd"], + ); + for (const key of [ + "inputTokens", + "outputTokens", + "cacheReadTokens", + "cacheWriteTokens", + ] as const) { + nonNegativeNumber(usage[key], `${path}.${key}`, true); + } + if (usage.reasoningTokens !== undefined) { + nonNegativeNumber(usage.reasoningTokens, `${path}.reasoningTokens`, true); + } + if (usage.costUsd !== undefined) nonNegativeNumber(usage.costUsd, `${path}.costUsd`); +} + +function validateResponse(value: unknown, path: string): void { + const response = object(value, path); + exact( + response, + path, + ["providerId", "requestedModelId"], + ["routedModelId", "responseId", "nativeStopReason", "latencyMs"], + ); + string(response.providerId, `${path}.providerId`); + string(response.requestedModelId, `${path}.requestedModelId`); + for (const key of ["routedModelId", "responseId", "nativeStopReason"] as const) { + if (response[key] !== undefined) string(response[key], `${path}.${key}`); + } + if (response.latencyMs !== undefined) nonNegativeNumber(response.latencyMs, `${path}.latencyMs`); +} + +function validateDiagnostics(value: unknown, path: string): void { + if (!Array.isArray(value)) fail(path, "must be an array"); + for (const [index, item] of value.entries()) { + const itemPath = `${path}[${index}]`; + const diagnostic = object(item, itemPath); + exact(diagnostic, itemPath, ["code", "message", "severity"]); + string(diagnostic.code, `${itemPath}.code`); + string(diagnostic.message, `${itemPath}.message`); + if (!new Set(["info", "warning", "error"]).has(String(diagnostic.severity))) { + fail(`${itemPath}.severity`, "must be info, warning, or error"); + } + } +} + +function validateTerminalMetadata(event: Record, path: string): void { + if (event.partial !== undefined) boolean(event.partial, `${path}.partial`); + if (event.response !== undefined) validateResponse(event.response, `${path}.response`); + if (event.diagnostics !== undefined) + validateDiagnostics(event.diagnostics, `${path}.diagnostics`); +} + +/** Validates an untrusted provider event before it crosses into the kernel. */ +export function parseModelStreamEvent(value: unknown, path = "modelStreamEvent"): ModelStreamEvent { + const event = object(value, path); + const type = string(event.type, `${path}.type`); + if (type === "text_delta" || type === "thinking_delta") { + exact(event, path, ["type", "text"], ["contentIndex"]); + string(event.text, `${path}.text`, true); + optionalPosition(event, path); + } else if (type === "tool_call_start") { + exact(event, path, ["type", "contentIndex", "callId", "name"]); + nonNegativeNumber(event.contentIndex, `${path}.contentIndex`, true); + string(event.callId, `${path}.callId`); + string(event.name, `${path}.name`); + } else if (type === "tool_call_delta") { + exact(event, path, ["type", "contentIndex", "callId", "argumentsDelta"]); + nonNegativeNumber(event.contentIndex, `${path}.contentIndex`, true); + string(event.callId, `${path}.callId`); + string(event.argumentsDelta, `${path}.argumentsDelta`, true); + } else if (type === "tool_call") { + exact(event, path, ["type", "callId", "name", "input"], ["contentIndex"]); + string(event.callId, `${path}.callId`); + string(event.name, `${path}.name`); + object(event.input, `${path}.input`); + validateJson(event.input, `${path}.input`); + optionalPosition(event, path); + } else if (type === "replay_metadata") { + exact( + event, + path, + ["type", "target", "contentIndex", "providerId", "apiDialect", "modelId"], + ["callId", "signature", "redacted", "responseId", "itemId", "namespace"], + ); + if (!new Set(["thinking", "text", "tool_call"]).has(String(event.target))) { + fail(`${path}.target`, "must be thinking, text, or tool_call"); + } + nonNegativeNumber(event.contentIndex, `${path}.contentIndex`, true); + string(event.providerId, `${path}.providerId`); + string(event.apiDialect, `${path}.apiDialect`); + string(event.modelId, `${path}.modelId`); + for (const key of ["callId", "signature", "responseId", "itemId", "namespace"] as const) { + if (event[key] !== undefined) string(event[key], `${path}.${key}`); + } + if (event.redacted !== undefined && typeof event.redacted !== "boolean") { + fail(`${path}.redacted`, "must be a boolean"); + } + if (event.target !== "thinking" && event.redacted !== undefined) { + fail(`${path}.redacted`, "is allowed only for thinking replay metadata"); + } + if (event.redacted === true && event.signature === undefined) { + fail(`${path}.redacted`, "requires signed thinking replay metadata"); + } + if (event.target === "tool_call" && event.callId === undefined) { + fail(`${path}.callId`, "is required for tool_call replay metadata"); + } + if (event.target !== "tool_call" && event.callId !== undefined) { + fail(`${path}.callId`, "is allowed only for tool_call replay metadata"); + } + if ( + event.signature === undefined && + event.responseId === undefined && + event.itemId === undefined && + event.namespace === undefined + ) { + fail(path, "must contain replay data"); + } + } else if (type === "completed") { + exact(event, path, ["type", "stopReason", "usage"], ["partial", "response", "diagnostics"]); + if ( + !new Set(["stop", "length", "tool_use", "error", "aborted"]).has(String(event.stopReason)) + ) { + fail(`${path}.stopReason`, "is not recognized"); + } + validateUsage(event.usage, `${path}.usage`); + validateTerminalMetadata(event, path); + } else if (type === "error") { + exact( + event, + path, + ["type", "code", "message", "retryable"], + ["partial", "category", "requestPhase", "retryAfterMs", "retry", "response", "diagnostics"], + ); + string(event.code, `${path}.code`); + string(event.message, `${path}.message`); + boolean(event.retryable, `${path}.retryable`); + if ( + event.category !== undefined && + !new Set([ + "rate_limit", + "overloaded", + "network", + "timeout", + "authentication", + "authorization", + "invalid_request", + "context_limit", + "content_policy", + "provider_internal", + "stream_interrupted", + "unknown", + ]).has(String(event.category)) + ) { + fail(`${path}.category`, "is not recognized"); + } + if ( + event.requestPhase !== undefined && + !new Set(["before_dispatch", "awaiting_response", "streaming", "unknown"]).has( + String(event.requestPhase), + ) + ) { + fail(`${path}.requestPhase`, "is not recognized"); + } + if (event.retryAfterMs !== undefined) { + nonNegativeNumber(event.retryAfterMs, `${path}.retryAfterMs`, true); + } + validateTerminalMetadata(event, path); + if (event.retry !== undefined) { + const retry = object(event.retry, `${path}.retry`); + exact(retry, `${path}.retry`, [], ["retryAfterMs", "resetAtEpochMs"]); + if (retry.retryAfterMs !== undefined) { + nonNegativeNumber(retry.retryAfterMs, `${path}.retry.retryAfterMs`, true); + } + if (retry.resetAtEpochMs !== undefined) { + nonNegativeNumber(retry.resetAtEpochMs, `${path}.retry.resetAtEpochMs`, true); + } + if (retry.retryAfterMs === undefined && retry.resetAtEpochMs === undefined) { + fail(`${path}.retry`, "must contain retryAfterMs or resetAtEpochMs"); + } + } + } else if (type === "aborted") { + exact(event, path, ["type"], ["partial"]); + if (event.partial !== undefined) boolean(event.partial, `${path}.partial`); + } else { + fail(`${path}.type`, "is not recognized"); + } + return value as ModelStreamEvent; +} diff --git a/packages/protocol/src/provider-management.ts b/packages/protocol/src/provider-management.ts new file mode 100644 index 0000000..23b1b30 --- /dev/null +++ b/packages/protocol/src/provider-management.ts @@ -0,0 +1,648 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import { type JsonObject, ProtocolValidationError } from "./event-envelope.ts"; +import type { ThinkingLevel } from "./events.ts"; + +export type ProviderAuthMethod = "environment" | "file" | "oauth" | "ambient" | "keyless"; +export type ProviderLoginMethod = "api_key" | "oauth"; +export type ProviderAuthenticationPhase = + | "idle" + | "authorizing" + | "authenticated" + | "reauthentication_required" + | "logged_out"; + +export interface ProviderAuthenticationStatus { + readonly providerId: string; + readonly phase: ProviderAuthenticationPhase; + readonly method?: ProviderLoginMethod; + readonly source?: string; +} + +export interface ProviderCatalogStatus { + readonly refreshable: boolean; + readonly generation?: number; + readonly checkedAt?: number; + readonly updatedAt?: number; + readonly source?: { + readonly id: string; + readonly kind: "provider_api" | "entitlement" | "gateway"; + readonly revision?: string; + }; +} + +export interface ProviderModelCost { + readonly inputUsdPerMTok: number; + readonly outputUsdPerMTok: number; + readonly cacheReadUsdPerMTok?: number; + readonly cacheWriteUsdPerMTok?: number; +} + +export interface ProviderTextModel { + readonly providerId: string; + readonly modelId: string; + readonly displayName: string; + readonly apiDialect: string; + readonly capabilities: { + readonly toolUse: boolean; + readonly structuredOutput: boolean; + readonly imageInput: boolean; + }; + readonly reasoning: boolean; + readonly supportedThinkingLevels: readonly ThinkingLevel[]; + readonly contextWindow: number; + readonly maxOutputTokens: number; + readonly cost?: ProviderModelCost; + readonly availability: { + readonly status: "available" | "preview" | "deprecated" | "unavailable"; + readonly reason?: string; + }; +} + +export interface ProviderInventoryGroup { + readonly providerId: string; + readonly displayName: string; + readonly enabled: boolean; + readonly regionFamily?: string; + readonly region?: string; + readonly authMethods: readonly ProviderAuthMethod[]; + readonly loginMethods: readonly ProviderLoginMethod[]; + readonly authentication: ProviderAuthenticationStatus; + readonly catalog: ProviderCatalogStatus; + readonly models: readonly ProviderTextModel[]; + readonly catalogError?: { + readonly code: "catalog_failure"; + readonly message: string; + readonly action: "refresh_catalog" | "configure_provider"; + }; +} + +export interface ProviderListParams { + readonly providerId?: string; +} + +export interface ProviderListResult { + readonly providers: readonly ProviderInventoryGroup[]; +} + +export interface ProviderCatalogRefreshParams { + readonly providerId?: string; +} + +export interface ProviderCatalogRefreshResult { + readonly providers: readonly { + readonly providerId: string; + readonly status: "refreshed" | "not_modified" | "superseded" | "unsupported" | "failed"; + readonly modelCount: number; + readonly error?: { + readonly code: + | "catalog_refresh_failed" + | "authentication_required" + | "entitlement_required" + | "entitlement_exhausted"; + readonly message: string; + readonly action: "retry" | "login" | "configure_provider"; + }; + }[]; +} + +export interface ProviderAuthenticationStatusParams { + readonly providerId?: string; +} + +export interface ProviderAuthenticationStatusResult { + readonly providers: readonly ProviderAuthenticationStatus[]; +} + +export interface ProviderLoginParams { + readonly providerId: string; + readonly method: ProviderLoginMethod; +} + +export interface ProviderLogoutParams { + readonly providerId: string; +} + +export type ProviderLoginResult = ProviderAuthenticationStatus; +export type ProviderLogoutResult = ProviderAuthenticationStatus; + +export const PROVIDER_RPC_ERROR_CODES = [ + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "authentication_unavailable", + "catalog_refresh_unsupported", + "catalog_refresh_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", +] as const; + +export type ProviderRpcErrorCode = (typeof PROVIDER_RPC_ERROR_CODES)[number]; + +export type ProviderErrorCategory = + | "provider" + | "model" + | "authentication" + | "catalog" + | "entitlement" + | "region" + | "configuration"; + +export type ProviderErrorAction = + | "login" + | "logout_then_login" + | "refresh_catalog" + | "configure_provider" + | "select_model" + | "select_region" + | "retry"; + +export interface ProviderRpcErrorDetails extends JsonObject { + readonly category: ProviderErrorCategory; + readonly action: ProviderErrorAction; + readonly providerId?: string; + readonly modelId?: string; +} + +const PROVIDER_ID = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; +const PROTOCOL_ID = /^[a-z][a-z0-9]*(?:[._-][a-z0-9]+)*$/; +const AUTH_METHODS: readonly ProviderAuthMethod[] = [ + "environment", + "file", + "oauth", + "ambient", + "keyless", +]; +const LOGIN_METHODS: readonly ProviderLoginMethod[] = ["api_key", "oauth"]; +const AUTH_PHASES: readonly ProviderAuthenticationPhase[] = [ + "idle", + "authorizing", + "authenticated", + "reauthentication_required", + "logged_out", +]; +const THINKING_LEVELS: readonly ThinkingLevel[] = [ + "off", + "minimal", + "low", + "medium", + "high", + "xhigh", + "max", +]; + +function object(value: unknown, path: string): Record { + if (typeof value !== "object" || value === null || Array.isArray(value)) { + throw new ProtocolValidationError(path, "must be an object"); + } + return value as Record; +} + +function exact(value: Record, path: string, keys: readonly string[]): void { + for (const key of Object.keys(value)) { + if (!keys.includes(key)) throw new ProtocolValidationError(`${path}.${key}`, "is not allowed"); + } +} + +function text(value: unknown, path: string, maximum: number, allowEmpty = false): string { + if (typeof value !== "string" || (!allowEmpty && value.length === 0)) { + throw new ProtocolValidationError(path, "must be a non-empty string"); + } + if (new TextEncoder().encode(value).byteLength > maximum) { + throw new ProtocolValidationError(path, `must not exceed ${maximum} UTF-8 bytes`); + } + return value; +} + +function providerId(value: unknown, path: string): string { + const result = text(value, path, 128); + if (!PROVIDER_ID.test(result)) throw new ProtocolValidationError(path, "must be a provider ID"); + return result; +} + +function nonNegativeInteger(value: unknown, path: string): number { + if (!Number.isSafeInteger(value) || (value as number) < 0) { + throw new ProtocolValidationError(path, "must be a non-negative safe integer"); + } + return value as number; +} + +function finiteNonNegative(value: unknown, path: string): number { + if (typeof value !== "number" || !Number.isFinite(value) || value < 0) { + throw new ProtocolValidationError(path, "must be a finite non-negative number"); + } + return value; +} + +function uniqueEnumArray( + value: unknown, + path: string, + allowed: readonly Value[], +): readonly Value[] { + if (!Array.isArray(value) || value.length > allowed.length) { + throw new ProtocolValidationError(path, `must contain at most ${allowed.length} values`); + } + const result = value.map((item, index) => { + if (!allowed.includes(item as Value)) { + throw new ProtocolValidationError(`${path}[${index}]`, "is not supported"); + } + return item as Value; + }); + if (new Set(result).size !== result.length) { + throw new ProtocolValidationError(path, "must not contain duplicates"); + } + return result; +} + +export function parseProviderAuthenticationStatus( + value: unknown, + path = "providerAuthenticationStatus", +): ProviderAuthenticationStatus { + const status = object(value, path); + exact(status, path, ["providerId", "phase", "method", "source"]); + if (!AUTH_PHASES.includes(status.phase as ProviderAuthenticationPhase)) { + throw new ProtocolValidationError(`${path}.phase`, "is not a valid authentication phase"); + } + if ( + status.method !== undefined && + !LOGIN_METHODS.includes(status.method as ProviderLoginMethod) + ) { + throw new ProtocolValidationError(`${path}.method`, "is not a valid login method"); + } + return { + providerId: providerId(status.providerId, `${path}.providerId`), + phase: status.phase as ProviderAuthenticationPhase, + ...(status.method === undefined ? {} : { method: status.method as ProviderLoginMethod }), + ...(status.source === undefined ? {} : { source: text(status.source, `${path}.source`, 256) }), + }; +} + +function parseCatalogStatus(value: unknown, path: string): ProviderCatalogStatus { + const catalog = object(value, path); + exact(catalog, path, ["refreshable", "generation", "checkedAt", "updatedAt", "source"]); + if (typeof catalog.refreshable !== "boolean") { + throw new ProtocolValidationError(`${path}.refreshable`, "must be a boolean"); + } + let source: ProviderCatalogStatus["source"]; + if (catalog.source !== undefined) { + const candidate = object(catalog.source, `${path}.source`); + exact(candidate, `${path}.source`, ["id", "kind", "revision"]); + if ( + candidate.kind !== "provider_api" && + candidate.kind !== "entitlement" && + candidate.kind !== "gateway" + ) { + throw new ProtocolValidationError(`${path}.source.kind`, "is not a valid source kind"); + } + const id = text(candidate.id, `${path}.source.id`, 512); + if (!PROTOCOL_ID.test(id)) { + throw new ProtocolValidationError(`${path}.source.id`, "must be a protocol identifier"); + } + source = { + id, + kind: candidate.kind, + ...(candidate.revision === undefined + ? {} + : { revision: text(candidate.revision, `${path}.source.revision`, 512) }), + }; + } + return { + refreshable: catalog.refreshable, + ...(catalog.generation === undefined + ? {} + : { generation: nonNegativeInteger(catalog.generation, `${path}.generation`) }), + ...(catalog.checkedAt === undefined + ? {} + : { checkedAt: nonNegativeInteger(catalog.checkedAt, `${path}.checkedAt`) }), + ...(catalog.updatedAt === undefined + ? {} + : { updatedAt: nonNegativeInteger(catalog.updatedAt, `${path}.updatedAt`) }), + ...(source === undefined ? {} : { source }), + }; +} + +function parseCost(value: unknown, path: string): ProviderModelCost { + const cost = object(value, path); + exact(cost, path, [ + "inputUsdPerMTok", + "outputUsdPerMTok", + "cacheReadUsdPerMTok", + "cacheWriteUsdPerMTok", + ]); + return { + inputUsdPerMTok: finiteNonNegative(cost.inputUsdPerMTok, `${path}.inputUsdPerMTok`), + outputUsdPerMTok: finiteNonNegative(cost.outputUsdPerMTok, `${path}.outputUsdPerMTok`), + ...(cost.cacheReadUsdPerMTok === undefined + ? {} + : { + cacheReadUsdPerMTok: finiteNonNegative( + cost.cacheReadUsdPerMTok, + `${path}.cacheReadUsdPerMTok`, + ), + }), + ...(cost.cacheWriteUsdPerMTok === undefined + ? {} + : { + cacheWriteUsdPerMTok: finiteNonNegative( + cost.cacheWriteUsdPerMTok, + `${path}.cacheWriteUsdPerMTok`, + ), + }), + }; +} + +function parseTextModel(value: unknown, path: string): ProviderTextModel { + const model = object(value, path); + exact(model, path, [ + "providerId", + "modelId", + "displayName", + "apiDialect", + "capabilities", + "reasoning", + "supportedThinkingLevels", + "contextWindow", + "maxOutputTokens", + "cost", + "availability", + ]); + const capabilities = object(model.capabilities, `${path}.capabilities`); + exact(capabilities, `${path}.capabilities`, ["toolUse", "structuredOutput", "imageInput"]); + for (const field of ["toolUse", "structuredOutput", "imageInput"] as const) { + if (typeof capabilities[field] !== "boolean") { + throw new ProtocolValidationError(`${path}.capabilities.${field}`, "must be a boolean"); + } + } + if (typeof model.reasoning !== "boolean") { + throw new ProtocolValidationError(`${path}.reasoning`, "must be a boolean"); + } + const availability = object(model.availability, `${path}.availability`); + exact(availability, `${path}.availability`, ["status", "reason"]); + if ( + availability.status !== "available" && + availability.status !== "preview" && + availability.status !== "deprecated" && + availability.status !== "unavailable" + ) { + throw new ProtocolValidationError(`${path}.availability.status`, "is not valid"); + } + return { + providerId: providerId(model.providerId, `${path}.providerId`), + modelId: text(model.modelId, `${path}.modelId`, 512), + displayName: text(model.displayName, `${path}.displayName`, 512), + apiDialect: text(model.apiDialect, `${path}.apiDialect`, 128), + capabilities: { + toolUse: capabilities.toolUse as boolean, + structuredOutput: capabilities.structuredOutput as boolean, + imageInput: capabilities.imageInput as boolean, + }, + reasoning: model.reasoning, + supportedThinkingLevels: uniqueEnumArray( + model.supportedThinkingLevels, + `${path}.supportedThinkingLevels`, + THINKING_LEVELS, + ), + contextWindow: nonNegativeInteger(model.contextWindow, `${path}.contextWindow`), + maxOutputTokens: nonNegativeInteger(model.maxOutputTokens, `${path}.maxOutputTokens`), + ...(model.cost === undefined ? {} : { cost: parseCost(model.cost, `${path}.cost`) }), + availability: { + status: availability.status, + ...(availability.reason === undefined + ? {} + : { reason: text(availability.reason, `${path}.availability.reason`, 2_000) }), + }, + }; +} + +function parseInventoryGroup(value: unknown, path: string): ProviderInventoryGroup { + const group = object(value, path); + exact(group, path, [ + "providerId", + "displayName", + "enabled", + "regionFamily", + "region", + "authMethods", + "loginMethods", + "authentication", + "catalog", + "models", + "catalogError", + ]); + if (typeof group.enabled !== "boolean") { + throw new ProtocolValidationError(`${path}.enabled`, "must be a boolean"); + } + if (!Array.isArray(group.models) || group.models.length > 5_000) { + throw new ProtocolValidationError(`${path}.models`, "must contain at most 5000 models"); + } + const id = providerId(group.providerId, `${path}.providerId`); + const models = group.models.map((model, index) => + parseTextModel(model, `${path}.models[${index}]`), + ); + if (models.some((model) => model.providerId !== id)) { + throw new ProtocolValidationError(`${path}.models`, "must belong to the provider group"); + } + let catalogError: ProviderInventoryGroup["catalogError"]; + if (group.catalogError !== undefined) { + const error = object(group.catalogError, `${path}.catalogError`); + exact(error, `${path}.catalogError`, ["code", "message", "action"]); + if (error.code !== "catalog_failure") { + throw new ProtocolValidationError(`${path}.catalogError.code`, "must be catalog_failure"); + } + if (error.action !== "refresh_catalog" && error.action !== "configure_provider") { + throw new ProtocolValidationError(`${path}.catalogError.action`, "is not valid"); + } + catalogError = { + code: error.code, + message: text(error.message, `${path}.catalogError.message`, 2_000), + action: error.action, + }; + } + return { + providerId: id, + displayName: text(group.displayName, `${path}.displayName`, 256), + enabled: group.enabled, + ...(group.regionFamily === undefined + ? {} + : { regionFamily: text(group.regionFamily, `${path}.regionFamily`, 128) }), + ...(group.region === undefined ? {} : { region: text(group.region, `${path}.region`, 128) }), + authMethods: uniqueEnumArray(group.authMethods, `${path}.authMethods`, AUTH_METHODS), + loginMethods: uniqueEnumArray(group.loginMethods, `${path}.loginMethods`, LOGIN_METHODS), + authentication: parseProviderAuthenticationStatus( + group.authentication, + `${path}.authentication`, + ), + catalog: parseCatalogStatus(group.catalog, `${path}.catalog`), + models, + ...(catalogError === undefined ? {} : { catalogError }), + }; +} + +export function parseProviderListResult(value: unknown): ProviderListResult { + const result = object(value, "providerList"); + exact(result, "providerList", ["providers"]); + if (!Array.isArray(result.providers) || result.providers.length > 256) { + throw new ProtocolValidationError( + "providerList.providers", + "must contain at most 256 providers", + ); + } + const providers = result.providers.map((provider, index) => + parseInventoryGroup(provider, `providerList.providers[${index}]`), + ); + if (new Set(providers.map((provider) => provider.providerId)).size !== providers.length) { + throw new ProtocolValidationError( + "providerList.providers", + "must not contain duplicate providers", + ); + } + return { providers }; +} + +export function parseProviderCatalogRefreshResult(value: unknown): ProviderCatalogRefreshResult { + const result = object(value, "providerCatalogRefresh"); + exact(result, "providerCatalogRefresh", ["providers"]); + if (!Array.isArray(result.providers) || result.providers.length > 256) { + throw new ProtocolValidationError( + "providerCatalogRefresh.providers", + "must contain at most 256 providers", + ); + } + const providers = result.providers.map((value, index) => { + const path = `providerCatalogRefresh.providers[${index}]`; + const provider = object(value, path); + exact(provider, path, ["providerId", "status", "modelCount", "error"]); + if ( + provider.status !== "refreshed" && + provider.status !== "not_modified" && + provider.status !== "superseded" && + provider.status !== "unsupported" && + provider.status !== "failed" + ) { + throw new ProtocolValidationError(`${path}.status`, "is not valid"); + } + let error: ProviderCatalogRefreshResult["providers"][number]["error"]; + if (provider.error !== undefined) { + const candidate = object(provider.error, `${path}.error`); + exact(candidate, `${path}.error`, ["code", "message", "action"]); + if ( + candidate.code !== "catalog_refresh_failed" && + candidate.code !== "authentication_required" && + candidate.code !== "entitlement_required" && + candidate.code !== "entitlement_exhausted" + ) { + throw new ProtocolValidationError(`${path}.error.code`, "is not valid"); + } + if ( + candidate.action !== "retry" && + candidate.action !== "login" && + candidate.action !== "configure_provider" + ) { + throw new ProtocolValidationError(`${path}.error.action`, "is not valid"); + } + error = { + code: candidate.code, + message: text(candidate.message, `${path}.error.message`, 2_000), + action: candidate.action, + }; + } + if ((provider.status === "failed") !== (error !== undefined)) { + throw new ProtocolValidationError( + `${path}.error`, + "must be present only for failed refreshes", + ); + } + return { + providerId: providerId(provider.providerId, `${path}.providerId`), + status: provider.status as ProviderCatalogRefreshResult["providers"][number]["status"], + modelCount: nonNegativeInteger(provider.modelCount, `${path}.modelCount`), + ...(error === undefined ? {} : { error }), + }; + }); + return { providers }; +} + +export function parseProviderAuthenticationStatusResult( + value: unknown, +): ProviderAuthenticationStatusResult { + const result = object(value, "providerAuthenticationStatusResult"); + exact(result, "providerAuthenticationStatusResult", ["providers"]); + if (!Array.isArray(result.providers) || result.providers.length > 256) { + throw new ProtocolValidationError( + "providerAuthenticationStatusResult.providers", + "must contain at most 256 providers", + ); + } + return { + providers: result.providers.map((provider, index) => + parseProviderAuthenticationStatus( + provider, + `providerAuthenticationStatusResult.providers[${index}]`, + ), + ), + }; +} + +export function parseProviderIdParam(value: unknown, path: string): string { + return providerId(value, path); +} + +export function parseProviderLoginMethod(value: unknown, path: string): ProviderLoginMethod { + if (!LOGIN_METHODS.includes(value as ProviderLoginMethod)) { + throw new ProtocolValidationError(path, "must be api_key or oauth"); + } + return value as ProviderLoginMethod; +} + +export function isProviderRpcErrorCode(value: string): value is ProviderRpcErrorCode { + return (PROVIDER_RPC_ERROR_CODES as readonly string[]).includes(value); +} + +export function parseProviderRpcErrorDetails( + value: unknown, + path: string, +): ProviderRpcErrorDetails { + const details = object(value, path); + exact(details, path, ["category", "action", "providerId", "modelId"]); + const categories: readonly ProviderErrorCategory[] = [ + "provider", + "model", + "authentication", + "catalog", + "entitlement", + "region", + "configuration", + ]; + const actions: readonly ProviderErrorAction[] = [ + "login", + "logout_then_login", + "refresh_catalog", + "configure_provider", + "select_model", + "select_region", + "retry", + ]; + if (!categories.includes(details.category as ProviderErrorCategory)) { + throw new ProtocolValidationError(`${path}.category`, "is not valid"); + } + if (!actions.includes(details.action as ProviderErrorAction)) { + throw new ProtocolValidationError(`${path}.action`, "is not valid"); + } + return { + category: details.category as ProviderErrorCategory, + action: details.action as ProviderErrorAction, + ...(details.providerId === undefined + ? {} + : { providerId: providerId(details.providerId, `${path}.providerId`) }), + ...(details.modelId === undefined + ? {} + : { modelId: text(details.modelId, `${path}.modelId`, 512) }), + }; +} diff --git a/packages/protocol/src/version.ts b/packages/protocol/src/version.ts index 2d5c87c..4d75bdd 100644 --- a/packages/protocol/src/version.ts +++ b/packages/protocol/src/version.ts @@ -7,4 +7,4 @@ export const EVENT_FORMAT_VERSION = 1 as const; /** Version negotiated by local daemon clients. */ -export const WIRE_PROTOCOL_VERSION = 11 as const; +export const WIRE_PROTOCOL_VERSION = 12 as const; diff --git a/packages/protocol/src/wire.ts b/packages/protocol/src/wire.ts index 97ae82e..39075c6 100644 --- a/packages/protocol/src/wire.ts +++ b/packages/protocol/src/wire.ts @@ -22,6 +22,26 @@ import type { UserContent, } from "./events.ts"; import { parseBlobReference, parseEvent, parseUserContent } from "./events.ts"; +import { + isProviderRpcErrorCode, + parseProviderAuthenticationStatus, + parseProviderAuthenticationStatusResult, + parseProviderCatalogRefreshResult, + parseProviderIdParam, + parseProviderListResult, + parseProviderLoginMethod, + parseProviderRpcErrorDetails, + type ProviderAuthenticationStatusParams, + type ProviderAuthenticationStatusResult, + type ProviderCatalogRefreshParams, + type ProviderCatalogRefreshResult, + type ProviderListParams, + type ProviderListResult, + type ProviderLoginParams, + type ProviderLoginResult, + type ProviderLogoutParams, + type ProviderLogoutResult, +} from "./provider-management.ts"; import { type ModelRequestSettings, parseModelRequestSettings } from "./model-request.ts"; @@ -31,6 +51,7 @@ export const MAX_WIRE_MESSAGE_BYTES = 1024 * 1024; export type EventCursor = string; export interface SessionModelSelection { + readonly providerId?: string; readonly requestSettings?: ModelRequestSettings; readonly modelId?: string; readonly thinkingLevel?: ThinkingLevel; @@ -638,6 +659,11 @@ export const WIRE_CAPABILITIES = [ "session.workspace.status", "session.workspace.diff", "session.workspace.checkpoint", + "provider.list", + "provider.catalog.refresh", + "provider.auth.status", + "provider.auth.login", + "provider.auth.logout", ] as const satisfies readonly CapabilityId[]; export interface ClientIdentity { @@ -692,6 +718,26 @@ export interface RpcMethodMap { readonly params: RequestCancelParams; readonly result: RequestCancelResult; }; + readonly "provider.list": { + readonly params: ProviderListParams; + readonly result: ProviderListResult; + }; + readonly "provider.catalog.refresh": { + readonly params: ProviderCatalogRefreshParams; + readonly result: ProviderCatalogRefreshResult; + }; + readonly "provider.auth.status": { + readonly params: ProviderAuthenticationStatusParams; + readonly result: ProviderAuthenticationStatusResult; + }; + readonly "provider.auth.login": { + readonly params: ProviderLoginParams; + readonly result: ProviderLoginResult; + }; + readonly "provider.auth.logout": { + readonly params: ProviderLogoutParams; + readonly result: ProviderLogoutResult; + }; readonly "session.create": { readonly params: { readonly cwd: string } & SessionConfiguration; readonly result: SessionOpenResult; @@ -800,6 +846,7 @@ export interface RpcMethodMap { readonly "session.configure": { readonly params: { readonly sessionId: SessionId } & SessionConfiguration; readonly result: { + readonly providerId: string; readonly modelId: string; readonly requestedThinkingLevel: ThinkingLevel; readonly effectiveThinkingLevel: ThinkingLevel; @@ -1013,6 +1060,20 @@ export const RPC_ERROR_CODES = [ "blob_corrupt", "invalid_blob_range", "blob_read_failed", + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "authentication_unavailable", + "catalog_refresh_unsupported", + "catalog_refresh_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", ] as const; export type RpcErrorCode = (typeof RPC_ERROR_CODES)[number] | (string & {}); @@ -1355,6 +1416,8 @@ function sessionProfile(value: unknown, path: string): SessionProfile | undefine } function selection(params: Record, path: string): SessionSelection { + const providerId = + params.providerId === undefined ? undefined : string(params.providerId, `${path}.providerId`); const modelId = params.modelId === undefined ? undefined : string(params.modelId, `${path}.modelId`); const thinkingLevel = params.thinkingLevel; @@ -1370,6 +1433,7 @@ function selection(params: Record, path: string): SessionSelect } } return { + ...(providerId === undefined ? {} : { providerId }), ...(params.requestSettings === undefined ? {} : { @@ -1447,9 +1511,44 @@ export function parseWireRequest(value: unknown): WireRequest { params: { requestId: nonNegativeInteger(params.requestId, "request.params.requestId") }, }; } + if ( + method === "provider.list" || + method === "provider.catalog.refresh" || + method === "provider.auth.status" + ) { + exact(params, "request.params", ["providerId"]); + return { + ...base, + method, + params: + params.providerId === undefined + ? {} + : { providerId: parseProviderIdParam(params.providerId, "request.params.providerId") }, + } as WireRequest; + } + if (method === "provider.auth.login") { + exact(params, "request.params", ["providerId", "method"]); + return { + ...base, + method, + params: { + providerId: parseProviderIdParam(params.providerId, "request.params.providerId"), + method: parseProviderLoginMethod(params.method, "request.params.method"), + }, + }; + } + if (method === "provider.auth.logout") { + exact(params, "request.params", ["providerId"]); + return { + ...base, + method, + params: { providerId: parseProviderIdParam(params.providerId, "request.params.providerId") }, + }; + } if (method === "session.create") { exact(params, "request.params", [ "cwd", + "providerId", "modelId", "thinkingLevel", "requestSettings", @@ -1696,6 +1795,7 @@ export function parseWireRequest(value: unknown): WireRequest { if (method === "session.configure") { exact(params, "request.params", [ "sessionId", + "providerId", "modelId", "thinkingLevel", "requestSettings", @@ -1706,6 +1806,7 @@ export function parseWireRequest(value: unknown): WireRequest { const configured = selection(params, "request.params"); const profile = sessionProfile(params.profile, "request.params.profile"); if ( + configured.providerId === undefined && configured.requestSettings === undefined && configured.modelId === undefined && configured.thinkingLevel === undefined && @@ -1715,7 +1816,7 @@ export function parseWireRequest(value: unknown): WireRequest { ) { throw new ProtocolValidationError( "request.params", - "must include modelId, thinkingLevel, requestSettings, webFetch, webSearch, or profile", + "must include providerId, modelId, thinkingLevel, requestSettings, webFetch, webSearch, or profile", ); } return { @@ -2189,6 +2290,14 @@ export function parseRpcResult( parsed = {}; } else if (method === "request.cancel") { parsed = parseBooleanResult(value, path, "cancellationRequested"); + } else if (method === "provider.list") { + parsed = parseProviderListResult(value); + } else if (method === "provider.catalog.refresh") { + parsed = parseProviderCatalogRefreshResult(value); + } else if (method === "provider.auth.status") { + parsed = parseProviderAuthenticationStatusResult(value); + } else if (method === "provider.auth.login" || method === "provider.auth.logout") { + parsed = parseProviderAuthenticationStatus(value, path); } else if (method === "session.create" || method === "session.resume") { parsed = parseSessionOpenResult(value, path); } else if (method === "session.list") { @@ -2318,6 +2427,7 @@ export function parseRpcResult( } else if (method === "session.configure") { const result = object(value, path); exact(result, path, [ + "providerId", "modelId", "requestedThinkingLevel", "effectiveThinkingLevel", @@ -2352,6 +2462,7 @@ export function parseRpcResult( throw new ProtocolValidationError(`${path}.profile`, "is required"); } parsed = { + providerId: boundedString(result.providerId, `${path}.providerId`, 128), modelId: boundedString(result.modelId, `${path}.modelId`, 512), requestedThinkingLevel: result.requestedThinkingLevel, effectiveThinkingLevel: result.effectiveThinkingLevel, @@ -2497,6 +2608,11 @@ export const RPC_METHODS = [ "connection.initialize", "connection.ping", "request.cancel", + "provider.list", + "provider.catalog.refresh", + "provider.auth.status", + "provider.auth.login", + "provider.auth.logout", "session.create", "session.resume", "session.list", @@ -2581,8 +2697,70 @@ export const RPC_METHOD_ERROR_CODES = { ], "connection.ping": [], "request.cancel": [], - "session.create": ["invalid_cwd", ...MUTATION_ERRORS, "corrupt_session", "content_too_large"], - "session.resume": ["unknown_session", "corrupt_session", "event_migration_required"], + "provider.list": ["provider_not_found", "provider_disabled", "catalog_refresh_failed"], + "provider.catalog.refresh": [ + "provider_not_found", + "provider_disabled", + "catalog_refresh_unsupported", + "catalog_refresh_failed", + "authentication_required", + "authentication_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", + ], + "provider.auth.status": ["provider_not_found", "provider_disabled", "authentication_failed"], + "provider.auth.login": [ + "provider_not_found", + "provider_disabled", + "authentication_required", + "authentication_unavailable", + "authentication_failed", + "region_required", + "region_unsupported", + "provider_configuration_required", + ], + "provider.auth.logout": [ + "provider_not_found", + "provider_disabled", + "authentication_unavailable", + "authentication_failed", + ], + "session.create": [ + "invalid_cwd", + ...MUTATION_ERRORS, + "corrupt_session", + "content_too_large", + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", + ], + "session.resume": [ + "unknown_session", + "corrupt_session", + "event_migration_required", + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", + ], "session.list": ["invalid_cwd", "unknown_cursor"], "session.history": ["unknown_cursor", "snapshot_required", "event_migration_required"], "session.ack": ["unknown_subscription", "unknown_cursor", "snapshot_required"], @@ -2674,6 +2852,17 @@ export const RPC_METHOD_ERROR_CODES = { "operation_active", ...MUTATION_ERRORS, "content_too_large", + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", ], "session.configure": [ ...SESSION_BASE_ERRORS, @@ -2681,6 +2870,17 @@ export const RPC_METHOD_ERROR_CODES = { "operation_active", ...MUTATION_ERRORS, "content_too_large", + "provider_not_found", + "provider_disabled", + "model_not_found", + "model_unavailable", + "authentication_required", + "authentication_failed", + "entitlement_required", + "entitlement_exhausted", + "region_required", + "region_unsupported", + "provider_configuration_required", ], "session.interaction.respond": [ ...SESSION_BASE_ERRORS, @@ -2893,9 +3093,13 @@ export function parseServerMessage(value: unknown): ServerMessage { code, message: boundedText(error.message, "message.error.message", 4096), retryable: error.retryable, - ...(error.details === undefined - ? {} - : { details: parseJsonObject(error.details, "message.error.details") }), + ...(isProviderRpcErrorCode(code) + ? { + details: parseProviderRpcErrorDetails(error.details, "message.error.details"), + } + : error.details === undefined + ? {} + : { details: parseJsonObject(error.details, "message.error.details") }), }, }; } diff --git a/packages/protocol/test/fixtures/conformance.json b/packages/protocol/test/fixtures/conformance.json index 1fe83e3..813a894 100644 --- a/packages/protocol/test/fixtures/conformance.json +++ b/packages/protocol/test/fixtures/conformance.json @@ -2,7 +2,7 @@ "SPDX-FileCopyrightText": "2026 Hari Srinivasan", "SPDX-License-Identifier": "Apache-2.0", "_generated": "@generated by packages/protocol/scripts/generate-conformance.ts; do not edit.", - "wireVersion": 11, + "wireVersion": 12, "requests": [ { "kind": "request", @@ -40,16 +40,57 @@ { "kind": "request", "id": 5, + "method": "provider.list", + "params": {} + }, + { + "kind": "request", + "id": 6, + "method": "provider.catalog.refresh", + "params": { + "providerId": "provider-1" + } + }, + { + "kind": "request", + "id": 7, + "method": "provider.auth.status", + "params": { + "providerId": "provider-1" + } + }, + { + "kind": "request", + "id": 8, + "method": "provider.auth.login", + "params": { + "providerId": "provider-1", + "method": "oauth" + } + }, + { + "kind": "request", + "id": 9, + "method": "provider.auth.logout", + "params": { + "providerId": "provider-1" + } + }, + { + "kind": "request", + "id": 10, "method": "session.create", "params": { "cwd": "/workspace", + "providerId": "provider-1", + "modelId": "model-1", "profile": "standard" }, "idempotencyKey": "00000000-0000-4000-8000-000000000020" }, { "kind": "request", - "id": 6, + "id": 11, "method": "session.resume", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -57,7 +98,7 @@ }, { "kind": "request", - "id": 7, + "id": 12, "method": "session.list", "params": { "scope": "all_local", @@ -67,7 +108,7 @@ }, { "kind": "request", - "id": 8, + "id": 13, "method": "session.history", "params": { "snapshotId": "snapshot-1", @@ -76,7 +117,7 @@ }, { "kind": "request", - "id": 9, + "id": 14, "method": "session.ack", "params": { "subscriptionId": "subscription-1", @@ -85,7 +126,7 @@ }, { "kind": "request", - "id": 10, + "id": 15, "method": "session.unsubscribe", "params": { "subscriptionId": "subscription-1" @@ -93,7 +134,7 @@ }, { "kind": "request", - "id": 11, + "id": 16, "method": "session.fork", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -103,7 +144,7 @@ }, { "kind": "request", - "id": 12, + "id": 17, "method": "session.clone", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -112,7 +153,7 @@ }, { "kind": "request", - "id": 13, + "id": 18, "method": "session.export", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -121,7 +162,7 @@ }, { "kind": "request", - "id": 14, + "id": 19, "method": "session.import", "params": { "inputDirectory": "/tmp/axl-session", @@ -131,7 +172,7 @@ }, { "kind": "request", - "id": 15, + "id": 20, "method": "session.send", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -147,7 +188,7 @@ }, { "kind": "request", - "id": 16, + "id": 21, "method": "session.steer", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -161,7 +202,7 @@ }, { "kind": "request", - "id": 17, + "id": 22, "method": "session.followUp", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -175,7 +216,7 @@ }, { "kind": "request", - "id": 18, + "id": 23, "method": "session.compact", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -184,7 +225,7 @@ }, { "kind": "request", - "id": 19, + "id": 24, "method": "session.queue.enqueue", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -200,7 +241,7 @@ }, { "kind": "request", - "id": 20, + "id": 25, "method": "session.queue.requeue", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -211,7 +252,7 @@ }, { "kind": "request", - "id": 21, + "id": 26, "method": "session.shell", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -222,7 +263,7 @@ }, { "kind": "request", - "id": 22, + "id": 27, "method": "session.interrupt", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -231,7 +272,7 @@ }, { "kind": "request", - "id": 23, + "id": 28, "method": "session.reload", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -240,10 +281,11 @@ }, { "kind": "request", - "id": 24, + "id": 29, "method": "session.configure", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", + "providerId": "provider-1", "modelId": "model-1", "thinkingLevel": "medium", "requestSettings": { @@ -255,7 +297,7 @@ }, { "kind": "request", - "id": 25, + "id": 30, "method": "session.interaction.respond", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -269,7 +311,7 @@ }, { "kind": "request", - "id": 26, + "id": 31, "method": "session.subscribe", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -277,7 +319,7 @@ }, { "kind": "request", - "id": 27, + "id": 32, "method": "session.workspace.list", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -287,7 +329,7 @@ }, { "kind": "request", - "id": 28, + "id": 33, "method": "session.workspace.read", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -298,7 +340,7 @@ }, { "kind": "request", - "id": 29, + "id": 34, "method": "session.workspace.status", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -307,7 +349,7 @@ }, { "kind": "request", - "id": 30, + "id": 35, "method": "session.workspace.diff", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -319,7 +361,7 @@ }, { "kind": "request", - "id": 31, + "id": 36, "method": "session.workspace.checkpoint", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -328,7 +370,7 @@ }, { "kind": "request", - "id": 32, + "id": 37, "method": "session.blob.start", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -339,7 +381,7 @@ }, { "kind": "request", - "id": 33, + "id": 38, "method": "session.blob.chunk", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -350,7 +392,7 @@ }, { "kind": "request", - "id": 34, + "id": 39, "method": "session.blob.commit", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -359,7 +401,7 @@ }, { "kind": "request", - "id": 35, + "id": 40, "method": "session.blob.abort", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -368,7 +410,7 @@ }, { "kind": "request", - "id": 36, + "id": 41, "method": "session.blob.read", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -379,7 +421,7 @@ }, { "kind": "request", - "id": 37, + "id": 42, "method": "session.dispose", "params": { "sessionId": "123e4567-e89b-42d3-a456-426614174000" @@ -404,7 +446,7 @@ "result": { "attachmentId": "attachment-1", "daemonInstanceId": "daemon-1", - "wireVersion": 11, + "wireVersion": 12, "grantedCapabilities": ["session.create"], "scope": "local_control", "heartbeatIntervalMs": 20000, @@ -428,6 +470,114 @@ { "kind": "success", "id": 5, + "method": "provider.list", + "result": { + "providers": [ + { + "providerId": "provider-1", + "displayName": "Provider One", + "enabled": true, + "regionFamily": "provider", + "region": "global", + "authMethods": ["environment", "oauth"], + "loginMethods": ["api_key", "oauth"], + "authentication": { + "providerId": "provider-1", + "phase": "authenticated", + "method": "oauth", + "source": "OAuth" + }, + "catalog": { + "refreshable": true, + "generation": 1, + "checkedAt": 1, + "updatedAt": 1, + "source": { + "id": "provider.catalog", + "kind": "provider_api", + "revision": "v1" + } + }, + "models": [ + { + "providerId": "provider-1", + "modelId": "model-1", + "displayName": "Model One", + "apiDialect": "openai-chat", + "capabilities": { + "toolUse": true, + "structuredOutput": true, + "imageInput": false + }, + "reasoning": true, + "supportedThinkingLevels": ["off", "low", "medium", "high"], + "contextWindow": 128000, + "maxOutputTokens": 16384, + "cost": { + "inputUsdPerMTok": 1, + "outputUsdPerMTok": 2 + }, + "availability": { + "status": "available" + } + } + ] + } + ] + } + }, + { + "kind": "success", + "id": 6, + "method": "provider.catalog.refresh", + "result": { + "providers": [ + { + "providerId": "provider-1", + "status": "refreshed", + "modelCount": 1 + } + ] + } + }, + { + "kind": "success", + "id": 7, + "method": "provider.auth.status", + "result": { + "providers": [ + { + "providerId": "provider-1", + "phase": "authenticated", + "method": "oauth", + "source": "OAuth" + } + ] + } + }, + { + "kind": "success", + "id": 8, + "method": "provider.auth.login", + "result": { + "providerId": "provider-1", + "phase": "authenticated", + "method": "oauth", + "source": "OAuth" + } + }, + { + "kind": "success", + "id": 9, + "method": "provider.auth.logout", + "result": { + "providerId": "provider-1", + "phase": "logged_out" + } + }, + { + "kind": "success", + "id": 10, "method": "session.create", "result": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -440,7 +590,7 @@ }, { "kind": "success", - "id": 6, + "id": 11, "method": "session.resume", "result": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -453,7 +603,7 @@ }, { "kind": "success", - "id": 7, + "id": 12, "method": "session.list", "result": { "sessions": [ @@ -475,7 +625,7 @@ }, { "kind": "success", - "id": 8, + "id": 13, "method": "session.history", "result": { "snapshotId": "snapshot-1", @@ -500,7 +650,7 @@ }, { "kind": "success", - "id": 9, + "id": 14, "method": "session.ack", "result": { "cursor": "cursor-1" @@ -508,7 +658,7 @@ }, { "kind": "success", - "id": 10, + "id": 15, "method": "session.unsubscribe", "result": { "unsubscribed": true @@ -516,7 +666,7 @@ }, { "kind": "success", - "id": 11, + "id": 16, "method": "session.fork", "result": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -530,7 +680,7 @@ }, { "kind": "success", - "id": 12, + "id": 17, "method": "session.clone", "result": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -543,7 +693,7 @@ }, { "kind": "success", - "id": 13, + "id": 18, "method": "session.export", "result": { "outputDirectory": "/tmp/axl-session", @@ -554,7 +704,7 @@ }, { "kind": "success", - "id": 14, + "id": 19, "method": "session.import", "result": { "sessionId": "123e4567-e89b-42d3-a456-426614174000", @@ -567,7 +717,7 @@ }, { "kind": "success", - "id": 15, + "id": 20, "method": "session.send", "result": { "operationId": "00000000-0000-4000-8000-000000000010", @@ -576,7 +726,7 @@ }, { "kind": "success", - "id": 16, + "id": 21, "method": "session.steer", "result": { "queued": true @@ -584,7 +734,7 @@ }, { "kind": "success", - "id": 17, + "id": 22, "method": "session.followUp", "result": { "queued": true @@ -592,7 +742,7 @@ }, { "kind": "success", - "id": 18, + "id": 23, "method": "session.compact", "result": { "eventId": "00000000-0000-4000-8000-000000000001" @@ -600,7 +750,7 @@ }, { "kind": "success", - "id": 19, + "id": 24, "method": "session.queue.enqueue", "result": { "queueItemId": "00000000-0000-4000-8000-000000000001", @@ -609,7 +759,7 @@ }, { "kind": "success", - "id": 20, + "id": 25, "method": "session.queue.requeue", "result": { "queueItemId": "00000000-0000-4000-8000-000000000001", @@ -618,7 +768,7 @@ }, { "kind": "success", - "id": 21, + "id": 26, "method": "session.shell", "result": { "operationId": "00000000-0000-4000-8000-000000000010", @@ -628,7 +778,7 @@ }, { "kind": "success", - "id": 22, + "id": 27, "method": "session.interrupt", "result": { "interrupted": true, @@ -637,7 +787,7 @@ }, { "kind": "success", - "id": 23, + "id": 28, "method": "session.reload", "result": { "boundaryEventIds": ["00000000-0000-4000-8000-000000000001"] @@ -645,9 +795,10 @@ }, { "kind": "success", - "id": 24, + "id": 29, "method": "session.configure", "result": { + "providerId": "provider-1", "modelId": "model-1", "requestedThinkingLevel": "medium", "effectiveThinkingLevel": "medium", @@ -663,7 +814,7 @@ }, { "kind": "success", - "id": 25, + "id": 30, "method": "session.interaction.respond", "result": { "interactionId": "interaction-1", @@ -672,7 +823,7 @@ }, { "kind": "success", - "id": 26, + "id": 31, "method": "session.subscribe", "result": { "subscriptionId": "subscription-1", @@ -704,7 +855,7 @@ }, { "kind": "success", - "id": 27, + "id": 32, "method": "session.workspace.list", "result": { "workspaceGeneration": "workspace-1", @@ -720,7 +871,7 @@ }, { "kind": "success", - "id": 28, + "id": 33, "method": "session.workspace.read", "result": { "workspaceGeneration": "workspace-1", @@ -736,7 +887,7 @@ }, { "kind": "success", - "id": 29, + "id": 34, "method": "session.workspace.status", "result": { "workspaceGeneration": "workspace-1", @@ -762,7 +913,7 @@ }, { "kind": "success", - "id": 30, + "id": 35, "method": "session.workspace.diff", "result": { "workspaceGeneration": "workspace-1", @@ -799,7 +950,7 @@ }, { "kind": "success", - "id": 31, + "id": 36, "method": "session.workspace.checkpoint", "result": { "enabled": true, @@ -808,7 +959,7 @@ }, { "kind": "success", - "id": 32, + "id": 37, "method": "session.blob.start", "result": { "uploadId": "upload-1", @@ -817,7 +968,7 @@ }, { "kind": "success", - "id": 33, + "id": 38, "method": "session.blob.chunk", "result": { "nextOffset": 4 @@ -825,7 +976,7 @@ }, { "kind": "success", - "id": 34, + "id": 39, "method": "session.blob.commit", "result": { "sha256": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", @@ -836,7 +987,7 @@ }, { "kind": "success", - "id": 35, + "id": 40, "method": "session.blob.abort", "result": { "aborted": true @@ -844,7 +995,7 @@ }, { "kind": "success", - "id": 36, + "id": 41, "method": "session.blob.read", "result": { "data": "YWJjZA==", @@ -855,7 +1006,7 @@ }, { "kind": "success", - "id": 37, + "id": 42, "method": "session.dispose", "result": { "disposed": true, @@ -1530,72 +1681,268 @@ "message": "Fixture error: blob_read_failed", "retryable": true } - } - ], - "allowedErrors": [ + }, { "kind": "error", - "id": 1, - "method": "daemon.info", + "id": 68, + "method": "provider.list", "error": { - "code": "daemon_stopping", - "message": "Fixture daemon.info error: daemon_stopping", - "retryable": false + "code": "provider_not_found", + "message": "Fixture error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2, - "method": "daemon.info", + "id": 69, + "method": "provider.list", "error": { - "code": "bad_request", - "message": "Fixture daemon.info error: bad_request", - "retryable": false + "code": "provider_disabled", + "message": "Fixture error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 3, - "method": "daemon.info", + "id": 70, + "method": "session.create", "error": { - "code": "not_initialized", - "message": "Fixture daemon.info error: not_initialized", - "retryable": false + "code": "model_not_found", + "message": "Fixture error: model_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 4, - "method": "daemon.info", + "id": 71, + "method": "session.create", "error": { - "code": "unsupported_capability", - "message": "Fixture daemon.info error: unsupported_capability", - "retryable": false + "code": "model_unavailable", + "message": "Fixture error: model_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 5, - "method": "daemon.info", + "id": 72, + "method": "provider.catalog.refresh", "error": { - "code": "rate_limited", - "message": "Fixture daemon.info error: rate_limited", - "retryable": true + "code": "authentication_required", + "message": "Fixture error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 6, - "method": "daemon.info", + "id": 73, + "method": "provider.catalog.refresh", "error": { - "code": "internal_error", - "message": "Fixture daemon.info error: internal_error", - "retryable": false + "code": "authentication_failed", + "message": "Fixture error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 7, + "id": 74, + "method": "provider.auth.login", + "error": { + "code": "authentication_unavailable", + "message": "Fixture error: authentication_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 75, + "method": "provider.catalog.refresh", + "error": { + "code": "catalog_refresh_unsupported", + "message": "Fixture error: catalog_refresh_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 76, + "method": "provider.list", + "error": { + "code": "catalog_refresh_failed", + "message": "Fixture error: catalog_refresh_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 77, + "method": "provider.catalog.refresh", + "error": { + "code": "entitlement_required", + "message": "Fixture error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 78, + "method": "provider.catalog.refresh", + "error": { + "code": "entitlement_exhausted", + "message": "Fixture error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 79, + "method": "provider.catalog.refresh", + "error": { + "code": "region_required", + "message": "Fixture error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 80, + "method": "provider.catalog.refresh", + "error": { + "code": "region_unsupported", + "message": "Fixture error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 81, + "method": "provider.catalog.refresh", + "error": { + "code": "provider_configuration_required", + "message": "Fixture error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + } + ], + "allowedErrors": [ + { + "kind": "error", + "id": 1, + "method": "daemon.info", + "error": { + "code": "daemon_stopping", + "message": "Fixture daemon.info error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 2, + "method": "daemon.info", + "error": { + "code": "bad_request", + "message": "Fixture daemon.info error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 3, + "method": "daemon.info", + "error": { + "code": "not_initialized", + "message": "Fixture daemon.info error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 4, + "method": "daemon.info", + "error": { + "code": "unsupported_capability", + "message": "Fixture daemon.info error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 5, + "method": "daemon.info", + "error": { + "code": "rate_limited", + "message": "Fixture daemon.info error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 6, + "method": "daemon.info", + "error": { + "code": "internal_error", + "message": "Fixture daemon.info error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 7, "method": "daemon.info", "error": { "code": "cancelled", @@ -1856,306 +2203,1370 @@ { "kind": "error", "id": 401, - "method": "session.create", + "method": "provider.list", "error": { "code": "daemon_stopping", - "message": "Fixture session.create error: daemon_stopping", + "message": "Fixture provider.list error: daemon_stopping", "retryable": false } }, { "kind": "error", "id": 402, - "method": "session.create", + "method": "provider.list", "error": { "code": "bad_request", - "message": "Fixture session.create error: bad_request", + "message": "Fixture provider.list error: bad_request", "retryable": false } }, { "kind": "error", "id": 403, - "method": "session.create", + "method": "provider.list", "error": { "code": "not_initialized", - "message": "Fixture session.create error: not_initialized", + "message": "Fixture provider.list error: not_initialized", "retryable": false } }, { "kind": "error", "id": 404, - "method": "session.create", + "method": "provider.list", "error": { "code": "unsupported_capability", - "message": "Fixture session.create error: unsupported_capability", + "message": "Fixture provider.list error: unsupported_capability", "retryable": false } }, { "kind": "error", "id": 405, - "method": "session.create", + "method": "provider.list", "error": { "code": "rate_limited", - "message": "Fixture session.create error: rate_limited", + "message": "Fixture provider.list error: rate_limited", "retryable": true } }, { "kind": "error", "id": 406, - "method": "session.create", + "method": "provider.list", "error": { "code": "internal_error", - "message": "Fixture session.create error: internal_error", + "message": "Fixture provider.list error: internal_error", "retryable": false } }, { "kind": "error", "id": 407, - "method": "session.create", + "method": "provider.list", "error": { "code": "cancelled", - "message": "Fixture session.create error: cancelled", + "message": "Fixture provider.list error: cancelled", "retryable": false } }, { "kind": "error", "id": 408, - "method": "session.create", + "method": "provider.list", "error": { - "code": "invalid_cwd", - "message": "Fixture session.create error: invalid_cwd", - "retryable": false + "code": "provider_not_found", + "message": "Fixture provider.list error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", "id": 409, - "method": "session.create", + "method": "provider.list", "error": { - "code": "invalid_idempotency_key", - "message": "Fixture session.create error: invalid_idempotency_key", - "retryable": false + "code": "provider_disabled", + "message": "Fixture provider.list error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", "id": 410, - "method": "session.create", - "error": { - "code": "idempotency_conflict", - "message": "Fixture session.create error: idempotency_conflict", - "retryable": false - } - }, - { - "kind": "error", - "id": 411, - "method": "session.create", - "error": { - "code": "corrupt_session", - "message": "Fixture session.create error: corrupt_session", - "retryable": false - } - }, - { - "kind": "error", - "id": 412, - "method": "session.create", + "method": "provider.list", "error": { - "code": "content_too_large", - "message": "Fixture session.create error: content_too_large", - "retryable": false + "code": "catalog_refresh_failed", + "message": "Fixture provider.list error: catalog_refresh_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", "id": 501, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "daemon_stopping", - "message": "Fixture session.resume error: daemon_stopping", + "message": "Fixture provider.catalog.refresh error: daemon_stopping", "retryable": false } }, { "kind": "error", "id": 502, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "bad_request", - "message": "Fixture session.resume error: bad_request", + "message": "Fixture provider.catalog.refresh error: bad_request", "retryable": false } }, { "kind": "error", "id": 503, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "not_initialized", - "message": "Fixture session.resume error: not_initialized", + "message": "Fixture provider.catalog.refresh error: not_initialized", "retryable": false } }, { "kind": "error", "id": 504, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "unsupported_capability", - "message": "Fixture session.resume error: unsupported_capability", + "message": "Fixture provider.catalog.refresh error: unsupported_capability", "retryable": false } }, { "kind": "error", "id": 505, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "rate_limited", - "message": "Fixture session.resume error: rate_limited", + "message": "Fixture provider.catalog.refresh error: rate_limited", "retryable": true } }, { "kind": "error", "id": 506, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "internal_error", - "message": "Fixture session.resume error: internal_error", + "message": "Fixture provider.catalog.refresh error: internal_error", "retryable": false } }, { "kind": "error", "id": 507, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { "code": "cancelled", - "message": "Fixture session.resume error: cancelled", + "message": "Fixture provider.catalog.refresh error: cancelled", "retryable": false } }, { "kind": "error", "id": 508, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { - "code": "unknown_session", - "message": "Fixture session.resume error: unknown_session", - "retryable": false + "code": "provider_not_found", + "message": "Fixture provider.catalog.refresh error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", "id": 509, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { - "code": "corrupt_session", - "message": "Fixture session.resume error: corrupt_session", - "retryable": false + "code": "provider_disabled", + "message": "Fixture provider.catalog.refresh error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", "id": 510, - "method": "session.resume", + "method": "provider.catalog.refresh", "error": { - "code": "event_migration_required", - "message": "Fixture session.resume error: event_migration_required", - "retryable": false + "code": "catalog_refresh_unsupported", + "message": "Fixture provider.catalog.refresh error: catalog_refresh_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 601, - "method": "session.list", + "id": 511, + "method": "provider.catalog.refresh", "error": { - "code": "daemon_stopping", - "message": "Fixture session.list error: daemon_stopping", - "retryable": false + "code": "catalog_refresh_failed", + "message": "Fixture provider.catalog.refresh error: catalog_refresh_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 602, - "method": "session.list", + "id": 512, + "method": "provider.catalog.refresh", "error": { - "code": "bad_request", - "message": "Fixture session.list error: bad_request", - "retryable": false + "code": "authentication_required", + "message": "Fixture provider.catalog.refresh error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 603, - "method": "session.list", + "id": 513, + "method": "provider.catalog.refresh", "error": { - "code": "not_initialized", - "message": "Fixture session.list error: not_initialized", - "retryable": false + "code": "authentication_failed", + "message": "Fixture provider.catalog.refresh error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 604, - "method": "session.list", + "id": 514, + "method": "provider.catalog.refresh", "error": { - "code": "unsupported_capability", - "message": "Fixture session.list error: unsupported_capability", - "retryable": false + "code": "entitlement_required", + "message": "Fixture provider.catalog.refresh error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 605, - "method": "session.list", + "id": 515, + "method": "provider.catalog.refresh", "error": { - "code": "rate_limited", - "message": "Fixture session.list error: rate_limited", - "retryable": true + "code": "entitlement_exhausted", + "message": "Fixture provider.catalog.refresh error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 606, - "method": "session.list", + "id": 516, + "method": "provider.catalog.refresh", "error": { - "code": "internal_error", - "message": "Fixture session.list error: internal_error", - "retryable": false + "code": "region_required", + "message": "Fixture provider.catalog.refresh error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 607, - "method": "session.list", + "id": 517, + "method": "provider.catalog.refresh", "error": { - "code": "cancelled", - "message": "Fixture session.list error: cancelled", - "retryable": false + "code": "region_unsupported", + "message": "Fixture provider.catalog.refresh error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 608, - "method": "session.list", + "id": 518, + "method": "provider.catalog.refresh", "error": { - "code": "invalid_cwd", - "message": "Fixture session.list error: invalid_cwd", - "retryable": false - } + "code": "provider_configuration_required", + "message": "Fixture provider.catalog.refresh error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 601, + "method": "provider.auth.status", + "error": { + "code": "daemon_stopping", + "message": "Fixture provider.auth.status error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 602, + "method": "provider.auth.status", + "error": { + "code": "bad_request", + "message": "Fixture provider.auth.status error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 603, + "method": "provider.auth.status", + "error": { + "code": "not_initialized", + "message": "Fixture provider.auth.status error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 604, + "method": "provider.auth.status", + "error": { + "code": "unsupported_capability", + "message": "Fixture provider.auth.status error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 605, + "method": "provider.auth.status", + "error": { + "code": "rate_limited", + "message": "Fixture provider.auth.status error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 606, + "method": "provider.auth.status", + "error": { + "code": "internal_error", + "message": "Fixture provider.auth.status error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 607, + "method": "provider.auth.status", + "error": { + "code": "cancelled", + "message": "Fixture provider.auth.status error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 608, + "method": "provider.auth.status", + "error": { + "code": "provider_not_found", + "message": "Fixture provider.auth.status error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 609, + "method": "provider.auth.status", + "error": { + "code": "provider_disabled", + "message": "Fixture provider.auth.status error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 610, + "method": "provider.auth.status", + "error": { + "code": "authentication_failed", + "message": "Fixture provider.auth.status error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 701, + "method": "provider.auth.login", + "error": { + "code": "daemon_stopping", + "message": "Fixture provider.auth.login error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 702, + "method": "provider.auth.login", + "error": { + "code": "bad_request", + "message": "Fixture provider.auth.login error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 703, + "method": "provider.auth.login", + "error": { + "code": "not_initialized", + "message": "Fixture provider.auth.login error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 704, + "method": "provider.auth.login", + "error": { + "code": "unsupported_capability", + "message": "Fixture provider.auth.login error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 705, + "method": "provider.auth.login", + "error": { + "code": "rate_limited", + "message": "Fixture provider.auth.login error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 706, + "method": "provider.auth.login", + "error": { + "code": "internal_error", + "message": "Fixture provider.auth.login error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 707, + "method": "provider.auth.login", + "error": { + "code": "cancelled", + "message": "Fixture provider.auth.login error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 708, + "method": "provider.auth.login", + "error": { + "code": "provider_not_found", + "message": "Fixture provider.auth.login error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 709, + "method": "provider.auth.login", + "error": { + "code": "provider_disabled", + "message": "Fixture provider.auth.login error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 710, + "method": "provider.auth.login", + "error": { + "code": "authentication_required", + "message": "Fixture provider.auth.login error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 711, + "method": "provider.auth.login", + "error": { + "code": "authentication_unavailable", + "message": "Fixture provider.auth.login error: authentication_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 712, + "method": "provider.auth.login", + "error": { + "code": "authentication_failed", + "message": "Fixture provider.auth.login error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 713, + "method": "provider.auth.login", + "error": { + "code": "region_required", + "message": "Fixture provider.auth.login error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 714, + "method": "provider.auth.login", + "error": { + "code": "region_unsupported", + "message": "Fixture provider.auth.login error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 715, + "method": "provider.auth.login", + "error": { + "code": "provider_configuration_required", + "message": "Fixture provider.auth.login error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 801, + "method": "provider.auth.logout", + "error": { + "code": "daemon_stopping", + "message": "Fixture provider.auth.logout error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 802, + "method": "provider.auth.logout", + "error": { + "code": "bad_request", + "message": "Fixture provider.auth.logout error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 803, + "method": "provider.auth.logout", + "error": { + "code": "not_initialized", + "message": "Fixture provider.auth.logout error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 804, + "method": "provider.auth.logout", + "error": { + "code": "unsupported_capability", + "message": "Fixture provider.auth.logout error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 805, + "method": "provider.auth.logout", + "error": { + "code": "rate_limited", + "message": "Fixture provider.auth.logout error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 806, + "method": "provider.auth.logout", + "error": { + "code": "internal_error", + "message": "Fixture provider.auth.logout error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 807, + "method": "provider.auth.logout", + "error": { + "code": "cancelled", + "message": "Fixture provider.auth.logout error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 808, + "method": "provider.auth.logout", + "error": { + "code": "provider_not_found", + "message": "Fixture provider.auth.logout error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 809, + "method": "provider.auth.logout", + "error": { + "code": "provider_disabled", + "message": "Fixture provider.auth.logout error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 810, + "method": "provider.auth.logout", + "error": { + "code": "authentication_unavailable", + "message": "Fixture provider.auth.logout error: authentication_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 811, + "method": "provider.auth.logout", + "error": { + "code": "authentication_failed", + "message": "Fixture provider.auth.logout error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 901, + "method": "session.create", + "error": { + "code": "daemon_stopping", + "message": "Fixture session.create error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 902, + "method": "session.create", + "error": { + "code": "bad_request", + "message": "Fixture session.create error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 903, + "method": "session.create", + "error": { + "code": "not_initialized", + "message": "Fixture session.create error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 904, + "method": "session.create", + "error": { + "code": "unsupported_capability", + "message": "Fixture session.create error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 905, + "method": "session.create", + "error": { + "code": "rate_limited", + "message": "Fixture session.create error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 906, + "method": "session.create", + "error": { + "code": "internal_error", + "message": "Fixture session.create error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 907, + "method": "session.create", + "error": { + "code": "cancelled", + "message": "Fixture session.create error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 908, + "method": "session.create", + "error": { + "code": "invalid_cwd", + "message": "Fixture session.create error: invalid_cwd", + "retryable": false + } + }, + { + "kind": "error", + "id": 909, + "method": "session.create", + "error": { + "code": "invalid_idempotency_key", + "message": "Fixture session.create error: invalid_idempotency_key", + "retryable": false + } + }, + { + "kind": "error", + "id": 910, + "method": "session.create", + "error": { + "code": "idempotency_conflict", + "message": "Fixture session.create error: idempotency_conflict", + "retryable": false + } + }, + { + "kind": "error", + "id": 911, + "method": "session.create", + "error": { + "code": "corrupt_session", + "message": "Fixture session.create error: corrupt_session", + "retryable": false + } + }, + { + "kind": "error", + "id": 912, + "method": "session.create", + "error": { + "code": "content_too_large", + "message": "Fixture session.create error: content_too_large", + "retryable": false + } + }, + { + "kind": "error", + "id": 913, + "method": "session.create", + "error": { + "code": "provider_not_found", + "message": "Fixture session.create error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 914, + "method": "session.create", + "error": { + "code": "provider_disabled", + "message": "Fixture session.create error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 915, + "method": "session.create", + "error": { + "code": "model_not_found", + "message": "Fixture session.create error: model_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 916, + "method": "session.create", + "error": { + "code": "model_unavailable", + "message": "Fixture session.create error: model_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 917, + "method": "session.create", + "error": { + "code": "authentication_required", + "message": "Fixture session.create error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 918, + "method": "session.create", + "error": { + "code": "authentication_failed", + "message": "Fixture session.create error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 919, + "method": "session.create", + "error": { + "code": "entitlement_required", + "message": "Fixture session.create error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 920, + "method": "session.create", + "error": { + "code": "entitlement_exhausted", + "message": "Fixture session.create error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 921, + "method": "session.create", + "error": { + "code": "region_required", + "message": "Fixture session.create error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 922, + "method": "session.create", + "error": { + "code": "region_unsupported", + "message": "Fixture session.create error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 923, + "method": "session.create", + "error": { + "code": "provider_configuration_required", + "message": "Fixture session.create error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1001, + "method": "session.resume", + "error": { + "code": "daemon_stopping", + "message": "Fixture session.resume error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 1002, + "method": "session.resume", + "error": { + "code": "bad_request", + "message": "Fixture session.resume error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 1003, + "method": "session.resume", + "error": { + "code": "not_initialized", + "message": "Fixture session.resume error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 1004, + "method": "session.resume", + "error": { + "code": "unsupported_capability", + "message": "Fixture session.resume error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 1005, + "method": "session.resume", + "error": { + "code": "rate_limited", + "message": "Fixture session.resume error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 1006, + "method": "session.resume", + "error": { + "code": "internal_error", + "message": "Fixture session.resume error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 1007, + "method": "session.resume", + "error": { + "code": "cancelled", + "message": "Fixture session.resume error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 1008, + "method": "session.resume", + "error": { + "code": "unknown_session", + "message": "Fixture session.resume error: unknown_session", + "retryable": false + } + }, + { + "kind": "error", + "id": 1009, + "method": "session.resume", + "error": { + "code": "corrupt_session", + "message": "Fixture session.resume error: corrupt_session", + "retryable": false + } + }, + { + "kind": "error", + "id": 1010, + "method": "session.resume", + "error": { + "code": "event_migration_required", + "message": "Fixture session.resume error: event_migration_required", + "retryable": false + } + }, + { + "kind": "error", + "id": 1011, + "method": "session.resume", + "error": { + "code": "provider_not_found", + "message": "Fixture session.resume error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1012, + "method": "session.resume", + "error": { + "code": "provider_disabled", + "message": "Fixture session.resume error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1013, + "method": "session.resume", + "error": { + "code": "model_not_found", + "message": "Fixture session.resume error: model_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1014, + "method": "session.resume", + "error": { + "code": "model_unavailable", + "message": "Fixture session.resume error: model_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1015, + "method": "session.resume", + "error": { + "code": "authentication_required", + "message": "Fixture session.resume error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1016, + "method": "session.resume", + "error": { + "code": "authentication_failed", + "message": "Fixture session.resume error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1017, + "method": "session.resume", + "error": { + "code": "entitlement_required", + "message": "Fixture session.resume error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1018, + "method": "session.resume", + "error": { + "code": "entitlement_exhausted", + "message": "Fixture session.resume error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1019, + "method": "session.resume", + "error": { + "code": "region_required", + "message": "Fixture session.resume error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1020, + "method": "session.resume", + "error": { + "code": "region_unsupported", + "message": "Fixture session.resume error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1021, + "method": "session.resume", + "error": { + "code": "provider_configuration_required", + "message": "Fixture session.resume error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 1101, + "method": "session.list", + "error": { + "code": "daemon_stopping", + "message": "Fixture session.list error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 1102, + "method": "session.list", + "error": { + "code": "bad_request", + "message": "Fixture session.list error: bad_request", + "retryable": false + } }, { "kind": "error", - "id": 609, + "id": 1103, + "method": "session.list", + "error": { + "code": "not_initialized", + "message": "Fixture session.list error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 1104, + "method": "session.list", + "error": { + "code": "unsupported_capability", + "message": "Fixture session.list error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 1105, + "method": "session.list", + "error": { + "code": "rate_limited", + "message": "Fixture session.list error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 1106, + "method": "session.list", + "error": { + "code": "internal_error", + "message": "Fixture session.list error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 1107, + "method": "session.list", + "error": { + "code": "cancelled", + "message": "Fixture session.list error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 1108, + "method": "session.list", + "error": { + "code": "invalid_cwd", + "message": "Fixture session.list error: invalid_cwd", + "retryable": false + } + }, + { + "kind": "error", + "id": 1109, "method": "session.list", "error": { "code": "unknown_cursor", @@ -2165,7 +3576,7 @@ }, { "kind": "error", - "id": 701, + "id": 1201, "method": "session.history", "error": { "code": "daemon_stopping", @@ -2175,7 +3586,7 @@ }, { "kind": "error", - "id": 702, + "id": 1202, "method": "session.history", "error": { "code": "bad_request", @@ -2185,7 +3596,7 @@ }, { "kind": "error", - "id": 703, + "id": 1203, "method": "session.history", "error": { "code": "not_initialized", @@ -2195,7 +3606,7 @@ }, { "kind": "error", - "id": 704, + "id": 1204, "method": "session.history", "error": { "code": "unsupported_capability", @@ -2205,7 +3616,7 @@ }, { "kind": "error", - "id": 705, + "id": 1205, "method": "session.history", "error": { "code": "rate_limited", @@ -2215,7 +3626,7 @@ }, { "kind": "error", - "id": 706, + "id": 1206, "method": "session.history", "error": { "code": "internal_error", @@ -2225,7 +3636,7 @@ }, { "kind": "error", - "id": 707, + "id": 1207, "method": "session.history", "error": { "code": "cancelled", @@ -2235,7 +3646,7 @@ }, { "kind": "error", - "id": 708, + "id": 1208, "method": "session.history", "error": { "code": "unknown_cursor", @@ -2245,7 +3656,7 @@ }, { "kind": "error", - "id": 709, + "id": 1209, "method": "session.history", "error": { "code": "snapshot_required", @@ -2255,7 +3666,7 @@ }, { "kind": "error", - "id": 710, + "id": 1210, "method": "session.history", "error": { "code": "event_migration_required", @@ -2265,7 +3676,7 @@ }, { "kind": "error", - "id": 801, + "id": 1301, "method": "session.ack", "error": { "code": "daemon_stopping", @@ -2275,7 +3686,7 @@ }, { "kind": "error", - "id": 802, + "id": 1302, "method": "session.ack", "error": { "code": "bad_request", @@ -2285,7 +3696,7 @@ }, { "kind": "error", - "id": 803, + "id": 1303, "method": "session.ack", "error": { "code": "not_initialized", @@ -2295,7 +3706,7 @@ }, { "kind": "error", - "id": 804, + "id": 1304, "method": "session.ack", "error": { "code": "unsupported_capability", @@ -2305,7 +3716,7 @@ }, { "kind": "error", - "id": 805, + "id": 1305, "method": "session.ack", "error": { "code": "rate_limited", @@ -2315,7 +3726,7 @@ }, { "kind": "error", - "id": 806, + "id": 1306, "method": "session.ack", "error": { "code": "internal_error", @@ -2325,7 +3736,7 @@ }, { "kind": "error", - "id": 807, + "id": 1307, "method": "session.ack", "error": { "code": "cancelled", @@ -2335,7 +3746,7 @@ }, { "kind": "error", - "id": 808, + "id": 1308, "method": "session.ack", "error": { "code": "unknown_subscription", @@ -2345,7 +3756,7 @@ }, { "kind": "error", - "id": 809, + "id": 1309, "method": "session.ack", "error": { "code": "unknown_cursor", @@ -2355,7 +3766,7 @@ }, { "kind": "error", - "id": 810, + "id": 1310, "method": "session.ack", "error": { "code": "snapshot_required", @@ -2365,7 +3776,7 @@ }, { "kind": "error", - "id": 901, + "id": 1401, "method": "session.unsubscribe", "error": { "code": "daemon_stopping", @@ -2375,7 +3786,7 @@ }, { "kind": "error", - "id": 902, + "id": 1402, "method": "session.unsubscribe", "error": { "code": "bad_request", @@ -2385,7 +3796,7 @@ }, { "kind": "error", - "id": 903, + "id": 1403, "method": "session.unsubscribe", "error": { "code": "not_initialized", @@ -2395,7 +3806,7 @@ }, { "kind": "error", - "id": 904, + "id": 1404, "method": "session.unsubscribe", "error": { "code": "unsupported_capability", @@ -2405,7 +3816,7 @@ }, { "kind": "error", - "id": 905, + "id": 1405, "method": "session.unsubscribe", "error": { "code": "rate_limited", @@ -2415,7 +3826,7 @@ }, { "kind": "error", - "id": 906, + "id": 1406, "method": "session.unsubscribe", "error": { "code": "internal_error", @@ -2425,7 +3836,7 @@ }, { "kind": "error", - "id": 907, + "id": 1407, "method": "session.unsubscribe", "error": { "code": "cancelled", @@ -2435,7 +3846,7 @@ }, { "kind": "error", - "id": 908, + "id": 1408, "method": "session.unsubscribe", "error": { "code": "unknown_subscription", @@ -2445,7 +3856,7 @@ }, { "kind": "error", - "id": 1001, + "id": 1501, "method": "session.fork", "error": { "code": "daemon_stopping", @@ -2455,7 +3866,7 @@ }, { "kind": "error", - "id": 1002, + "id": 1502, "method": "session.fork", "error": { "code": "bad_request", @@ -2465,7 +3876,7 @@ }, { "kind": "error", - "id": 1003, + "id": 1503, "method": "session.fork", "error": { "code": "not_initialized", @@ -2475,7 +3886,7 @@ }, { "kind": "error", - "id": 1004, + "id": 1504, "method": "session.fork", "error": { "code": "unsupported_capability", @@ -2485,7 +3896,7 @@ }, { "kind": "error", - "id": 1005, + "id": 1505, "method": "session.fork", "error": { "code": "rate_limited", @@ -2495,7 +3906,7 @@ }, { "kind": "error", - "id": 1006, + "id": 1506, "method": "session.fork", "error": { "code": "internal_error", @@ -2505,7 +3916,7 @@ }, { "kind": "error", - "id": 1007, + "id": 1507, "method": "session.fork", "error": { "code": "cancelled", @@ -2515,7 +3926,7 @@ }, { "kind": "error", - "id": 1008, + "id": 1508, "method": "session.fork", "error": { "code": "unknown_session", @@ -2525,7 +3936,7 @@ }, { "kind": "error", - "id": 1009, + "id": 1509, "method": "session.fork", "error": { "code": "event_migration_required", @@ -2535,7 +3946,7 @@ }, { "kind": "error", - "id": 1010, + "id": 1510, "method": "session.fork", "error": { "code": "corrupt_session", @@ -2545,7 +3956,7 @@ }, { "kind": "error", - "id": 1011, + "id": 1511, "method": "session.fork", "error": { "code": "operation_active", @@ -2555,7 +3966,7 @@ }, { "kind": "error", - "id": 1012, + "id": 1512, "method": "session.fork", "error": { "code": "invalid_fork_point", @@ -2565,7 +3976,7 @@ }, { "kind": "error", - "id": 1013, + "id": 1513, "method": "session.fork", "error": { "code": "invalid_idempotency_key", @@ -2575,7 +3986,7 @@ }, { "kind": "error", - "id": 1014, + "id": 1514, "method": "session.fork", "error": { "code": "idempotency_conflict", @@ -2585,7 +3996,7 @@ }, { "kind": "error", - "id": 1015, + "id": 1515, "method": "session.fork", "error": { "code": "content_too_large", @@ -2595,7 +4006,7 @@ }, { "kind": "error", - "id": 1101, + "id": 1601, "method": "session.clone", "error": { "code": "daemon_stopping", @@ -2605,7 +4016,7 @@ }, { "kind": "error", - "id": 1102, + "id": 1602, "method": "session.clone", "error": { "code": "bad_request", @@ -2615,7 +4026,7 @@ }, { "kind": "error", - "id": 1103, + "id": 1603, "method": "session.clone", "error": { "code": "not_initialized", @@ -2625,7 +4036,7 @@ }, { "kind": "error", - "id": 1104, + "id": 1604, "method": "session.clone", "error": { "code": "unsupported_capability", @@ -2635,7 +4046,7 @@ }, { "kind": "error", - "id": 1105, + "id": 1605, "method": "session.clone", "error": { "code": "rate_limited", @@ -2645,7 +4056,7 @@ }, { "kind": "error", - "id": 1106, + "id": 1606, "method": "session.clone", "error": { "code": "internal_error", @@ -2655,7 +4066,7 @@ }, { "kind": "error", - "id": 1107, + "id": 1607, "method": "session.clone", "error": { "code": "cancelled", @@ -2665,7 +4076,7 @@ }, { "kind": "error", - "id": 1108, + "id": 1608, "method": "session.clone", "error": { "code": "unknown_session", @@ -2675,7 +4086,7 @@ }, { "kind": "error", - "id": 1109, + "id": 1609, "method": "session.clone", "error": { "code": "event_migration_required", @@ -2685,7 +4096,7 @@ }, { "kind": "error", - "id": 1110, + "id": 1610, "method": "session.clone", "error": { "code": "corrupt_session", @@ -2695,7 +4106,7 @@ }, { "kind": "error", - "id": 1111, + "id": 1611, "method": "session.clone", "error": { "code": "operation_active", @@ -2705,7 +4116,7 @@ }, { "kind": "error", - "id": 1112, + "id": 1612, "method": "session.clone", "error": { "code": "empty_session", @@ -2715,7 +4126,7 @@ }, { "kind": "error", - "id": 1113, + "id": 1613, "method": "session.clone", "error": { "code": "invalid_idempotency_key", @@ -2725,7 +4136,7 @@ }, { "kind": "error", - "id": 1114, + "id": 1614, "method": "session.clone", "error": { "code": "idempotency_conflict", @@ -2735,7 +4146,7 @@ }, { "kind": "error", - "id": 1115, + "id": 1615, "method": "session.clone", "error": { "code": "content_too_large", @@ -2745,7 +4156,7 @@ }, { "kind": "error", - "id": 1201, + "id": 1701, "method": "session.export", "error": { "code": "daemon_stopping", @@ -2755,7 +4166,7 @@ }, { "kind": "error", - "id": 1202, + "id": 1702, "method": "session.export", "error": { "code": "bad_request", @@ -2765,7 +4176,7 @@ }, { "kind": "error", - "id": 1203, + "id": 1703, "method": "session.export", "error": { "code": "not_initialized", @@ -2775,7 +4186,7 @@ }, { "kind": "error", - "id": 1204, + "id": 1704, "method": "session.export", "error": { "code": "unsupported_capability", @@ -2785,7 +4196,7 @@ }, { "kind": "error", - "id": 1205, + "id": 1705, "method": "session.export", "error": { "code": "rate_limited", @@ -2795,7 +4206,7 @@ }, { "kind": "error", - "id": 1206, + "id": 1706, "method": "session.export", "error": { "code": "internal_error", @@ -2805,7 +4216,7 @@ }, { "kind": "error", - "id": 1207, + "id": 1707, "method": "session.export", "error": { "code": "cancelled", @@ -2815,7 +4226,7 @@ }, { "kind": "error", - "id": 1208, + "id": 1708, "method": "session.export", "error": { "code": "unknown_session", @@ -2825,7 +4236,7 @@ }, { "kind": "error", - "id": 1209, + "id": 1709, "method": "session.export", "error": { "code": "event_migration_required", @@ -2835,7 +4246,7 @@ }, { "kind": "error", - "id": 1210, + "id": 1710, "method": "session.export", "error": { "code": "operation_active", @@ -2845,7 +4256,7 @@ }, { "kind": "error", - "id": 1211, + "id": 1711, "method": "session.export", "error": { "code": "invalid_path", @@ -2855,7 +4266,7 @@ }, { "kind": "error", - "id": 1212, + "id": 1712, "method": "session.export", "error": { "code": "artifact_exists", @@ -2865,7 +4276,7 @@ }, { "kind": "error", - "id": 1213, + "id": 1713, "method": "session.export", "error": { "code": "blob_missing", @@ -2875,7 +4286,7 @@ }, { "kind": "error", - "id": 1214, + "id": 1714, "method": "session.export", "error": { "code": "blob_corrupt", @@ -2885,7 +4296,7 @@ }, { "kind": "error", - "id": 1301, + "id": 1801, "method": "session.import", "error": { "code": "daemon_stopping", @@ -2895,7 +4306,7 @@ }, { "kind": "error", - "id": 1302, + "id": 1802, "method": "session.import", "error": { "code": "bad_request", @@ -2905,7 +4316,7 @@ }, { "kind": "error", - "id": 1303, + "id": 1803, "method": "session.import", "error": { "code": "not_initialized", @@ -2915,7 +4326,7 @@ }, { "kind": "error", - "id": 1304, + "id": 1804, "method": "session.import", "error": { "code": "unsupported_capability", @@ -2925,7 +4336,7 @@ }, { "kind": "error", - "id": 1305, + "id": 1805, "method": "session.import", "error": { "code": "rate_limited", @@ -2935,7 +4346,7 @@ }, { "kind": "error", - "id": 1306, + "id": 1806, "method": "session.import", "error": { "code": "internal_error", @@ -2945,7 +4356,7 @@ }, { "kind": "error", - "id": 1307, + "id": 1807, "method": "session.import", "error": { "code": "cancelled", @@ -2955,7 +4366,7 @@ }, { "kind": "error", - "id": 1308, + "id": 1808, "method": "session.import", "error": { "code": "invalid_cwd", @@ -2965,7 +4376,7 @@ }, { "kind": "error", - "id": 1309, + "id": 1809, "method": "session.import", "error": { "code": "invalid_path", @@ -2975,7 +4386,7 @@ }, { "kind": "error", - "id": 1310, + "id": 1810, "method": "session.import", "error": { "code": "not_found", @@ -2985,7 +4396,7 @@ }, { "kind": "error", - "id": 1311, + "id": 1811, "method": "session.import", "error": { "code": "invalid_artifact", @@ -2995,7 +4406,7 @@ }, { "kind": "error", - "id": 1312, + "id": 1812, "method": "session.import", "error": { "code": "corrupt_session", @@ -3005,7 +4416,7 @@ }, { "kind": "error", - "id": 1313, + "id": 1813, "method": "session.import", "error": { "code": "blob_missing", @@ -3015,7 +4426,7 @@ }, { "kind": "error", - "id": 1314, + "id": 1814, "method": "session.import", "error": { "code": "blob_corrupt", @@ -3025,7 +4436,7 @@ }, { "kind": "error", - "id": 1315, + "id": 1815, "method": "session.import", "error": { "code": "content_too_large", @@ -3035,7 +4446,7 @@ }, { "kind": "error", - "id": 1316, + "id": 1816, "method": "session.import", "error": { "code": "invalid_idempotency_key", @@ -3045,7 +4456,7 @@ }, { "kind": "error", - "id": 1317, + "id": 1817, "method": "session.import", "error": { "code": "idempotency_conflict", @@ -3055,7 +4466,7 @@ }, { "kind": "error", - "id": 1401, + "id": 1901, "method": "session.send", "error": { "code": "daemon_stopping", @@ -3065,7 +4476,7 @@ }, { "kind": "error", - "id": 1402, + "id": 1902, "method": "session.send", "error": { "code": "bad_request", @@ -3075,7 +4486,7 @@ }, { "kind": "error", - "id": 1403, + "id": 1903, "method": "session.send", "error": { "code": "not_initialized", @@ -3085,7 +4496,7 @@ }, { "kind": "error", - "id": 1404, + "id": 1904, "method": "session.send", "error": { "code": "unsupported_capability", @@ -3095,7 +4506,7 @@ }, { "kind": "error", - "id": 1405, + "id": 1905, "method": "session.send", "error": { "code": "rate_limited", @@ -3105,7 +4516,7 @@ }, { "kind": "error", - "id": 1406, + "id": 1906, "method": "session.send", "error": { "code": "internal_error", @@ -3115,7 +4526,7 @@ }, { "kind": "error", - "id": 1407, + "id": 1907, "method": "session.send", "error": { "code": "cancelled", @@ -3125,7 +4536,7 @@ }, { "kind": "error", - "id": 1408, + "id": 1908, "method": "session.send", "error": { "code": "unknown_session", @@ -3135,7 +4546,7 @@ }, { "kind": "error", - "id": 1409, + "id": 1909, "method": "session.send", "error": { "code": "event_migration_required", @@ -3145,7 +4556,7 @@ }, { "kind": "error", - "id": 1410, + "id": 1910, "method": "session.send", "error": { "code": "operation_active", @@ -3155,7 +4566,7 @@ }, { "kind": "error", - "id": 1411, + "id": 1911, "method": "session.send", "error": { "code": "invalid_idempotency_key", @@ -3165,7 +4576,7 @@ }, { "kind": "error", - "id": 1412, + "id": 1912, "method": "session.send", "error": { "code": "idempotency_conflict", @@ -3175,7 +4586,7 @@ }, { "kind": "error", - "id": 1413, + "id": 1913, "method": "session.send", "error": { "code": "blob_not_owned", @@ -3185,7 +4596,7 @@ }, { "kind": "error", - "id": 1414, + "id": 1914, "method": "session.send", "error": { "code": "blob_missing", @@ -3195,7 +4606,7 @@ }, { "kind": "error", - "id": 1415, + "id": 1915, "method": "session.send", "error": { "code": "blob_corrupt", @@ -3205,7 +4616,7 @@ }, { "kind": "error", - "id": 1416, + "id": 1916, "method": "session.send", "error": { "code": "content_too_large", @@ -3215,7 +4626,7 @@ }, { "kind": "error", - "id": 1501, + "id": 2001, "method": "session.steer", "error": { "code": "daemon_stopping", @@ -3225,7 +4636,7 @@ }, { "kind": "error", - "id": 1502, + "id": 2002, "method": "session.steer", "error": { "code": "bad_request", @@ -3235,7 +4646,7 @@ }, { "kind": "error", - "id": 1503, + "id": 2003, "method": "session.steer", "error": { "code": "not_initialized", @@ -3245,7 +4656,7 @@ }, { "kind": "error", - "id": 1504, + "id": 2004, "method": "session.steer", "error": { "code": "unsupported_capability", @@ -3255,7 +4666,7 @@ }, { "kind": "error", - "id": 1505, + "id": 2005, "method": "session.steer", "error": { "code": "rate_limited", @@ -3265,7 +4676,7 @@ }, { "kind": "error", - "id": 1506, + "id": 2006, "method": "session.steer", "error": { "code": "internal_error", @@ -3275,7 +4686,7 @@ }, { "kind": "error", - "id": 1507, + "id": 2007, "method": "session.steer", "error": { "code": "cancelled", @@ -3285,7 +4696,7 @@ }, { "kind": "error", - "id": 1508, + "id": 2008, "method": "session.steer", "error": { "code": "unknown_session", @@ -3295,7 +4706,7 @@ }, { "kind": "error", - "id": 1509, + "id": 2009, "method": "session.steer", "error": { "code": "event_migration_required", @@ -3305,7 +4716,7 @@ }, { "kind": "error", - "id": 1510, + "id": 2010, "method": "session.steer", "error": { "code": "operation_inactive", @@ -3315,7 +4726,7 @@ }, { "kind": "error", - "id": 1511, + "id": 2011, "method": "session.steer", "error": { "code": "blob_not_owned", @@ -3325,7 +4736,7 @@ }, { "kind": "error", - "id": 1512, + "id": 2012, "method": "session.steer", "error": { "code": "blob_missing", @@ -3335,7 +4746,7 @@ }, { "kind": "error", - "id": 1513, + "id": 2013, "method": "session.steer", "error": { "code": "blob_corrupt", @@ -3345,7 +4756,7 @@ }, { "kind": "error", - "id": 1601, + "id": 2101, "method": "session.followUp", "error": { "code": "daemon_stopping", @@ -3355,7 +4766,7 @@ }, { "kind": "error", - "id": 1602, + "id": 2102, "method": "session.followUp", "error": { "code": "bad_request", @@ -3365,7 +4776,7 @@ }, { "kind": "error", - "id": 1603, + "id": 2103, "method": "session.followUp", "error": { "code": "not_initialized", @@ -3375,7 +4786,7 @@ }, { "kind": "error", - "id": 1604, + "id": 2104, "method": "session.followUp", "error": { "code": "unsupported_capability", @@ -3385,7 +4796,7 @@ }, { "kind": "error", - "id": 1605, + "id": 2105, "method": "session.followUp", "error": { "code": "rate_limited", @@ -3395,7 +4806,7 @@ }, { "kind": "error", - "id": 1606, + "id": 2106, "method": "session.followUp", "error": { "code": "internal_error", @@ -3405,7 +4816,7 @@ }, { "kind": "error", - "id": 1607, + "id": 2107, "method": "session.followUp", "error": { "code": "cancelled", @@ -3415,7 +4826,7 @@ }, { "kind": "error", - "id": 1608, + "id": 2108, "method": "session.followUp", "error": { "code": "unknown_session", @@ -3425,7 +4836,7 @@ }, { "kind": "error", - "id": 1609, + "id": 2109, "method": "session.followUp", "error": { "code": "event_migration_required", @@ -3435,7 +4846,7 @@ }, { "kind": "error", - "id": 1610, + "id": 2110, "method": "session.followUp", "error": { "code": "operation_inactive", @@ -3445,7 +4856,7 @@ }, { "kind": "error", - "id": 1611, + "id": 2111, "method": "session.followUp", "error": { "code": "blob_not_owned", @@ -3455,7 +4866,7 @@ }, { "kind": "error", - "id": 1612, + "id": 2112, "method": "session.followUp", "error": { "code": "blob_missing", @@ -3465,7 +4876,7 @@ }, { "kind": "error", - "id": 1613, + "id": 2113, "method": "session.followUp", "error": { "code": "blob_corrupt", @@ -3475,7 +4886,7 @@ }, { "kind": "error", - "id": 1701, + "id": 2201, "method": "session.compact", "error": { "code": "daemon_stopping", @@ -3485,7 +4896,7 @@ }, { "kind": "error", - "id": 1702, + "id": 2202, "method": "session.compact", "error": { "code": "bad_request", @@ -3495,7 +4906,7 @@ }, { "kind": "error", - "id": 1703, + "id": 2203, "method": "session.compact", "error": { "code": "not_initialized", @@ -3505,7 +4916,7 @@ }, { "kind": "error", - "id": 1704, + "id": 2204, "method": "session.compact", "error": { "code": "unsupported_capability", @@ -3515,7 +4926,7 @@ }, { "kind": "error", - "id": 1705, + "id": 2205, "method": "session.compact", "error": { "code": "rate_limited", @@ -3525,7 +4936,7 @@ }, { "kind": "error", - "id": 1706, + "id": 2206, "method": "session.compact", "error": { "code": "internal_error", @@ -3535,7 +4946,7 @@ }, { "kind": "error", - "id": 1707, + "id": 2207, "method": "session.compact", "error": { "code": "cancelled", @@ -3545,7 +4956,7 @@ }, { "kind": "error", - "id": 1708, + "id": 2208, "method": "session.compact", "error": { "code": "unknown_session", @@ -3555,7 +4966,7 @@ }, { "kind": "error", - "id": 1709, + "id": 2209, "method": "session.compact", "error": { "code": "event_migration_required", @@ -3565,7 +4976,7 @@ }, { "kind": "error", - "id": 1710, + "id": 2210, "method": "session.compact", "error": { "code": "operation_active", @@ -3575,7 +4986,7 @@ }, { "kind": "error", - "id": 1711, + "id": 2211, "method": "session.compact", "error": { "code": "content_too_large", @@ -3585,7 +4996,7 @@ }, { "kind": "error", - "id": 1801, + "id": 2301, "method": "session.queue.enqueue", "error": { "code": "daemon_stopping", @@ -3595,7 +5006,7 @@ }, { "kind": "error", - "id": 1802, + "id": 2302, "method": "session.queue.enqueue", "error": { "code": "bad_request", @@ -3605,7 +5016,7 @@ }, { "kind": "error", - "id": 1803, + "id": 2303, "method": "session.queue.enqueue", "error": { "code": "not_initialized", @@ -3615,7 +5026,7 @@ }, { "kind": "error", - "id": 1804, + "id": 2304, "method": "session.queue.enqueue", "error": { "code": "unsupported_capability", @@ -3625,7 +5036,7 @@ }, { "kind": "error", - "id": 1805, + "id": 2305, "method": "session.queue.enqueue", "error": { "code": "rate_limited", @@ -3635,7 +5046,7 @@ }, { "kind": "error", - "id": 1806, + "id": 2306, "method": "session.queue.enqueue", "error": { "code": "internal_error", @@ -3645,7 +5056,7 @@ }, { "kind": "error", - "id": 1807, + "id": 2307, "method": "session.queue.enqueue", "error": { "code": "cancelled", @@ -3655,7 +5066,7 @@ }, { "kind": "error", - "id": 1808, + "id": 2308, "method": "session.queue.enqueue", "error": { "code": "unknown_session", @@ -3665,7 +5076,7 @@ }, { "kind": "error", - "id": 1809, + "id": 2309, "method": "session.queue.enqueue", "error": { "code": "event_migration_required", @@ -3675,7 +5086,7 @@ }, { "kind": "error", - "id": 1810, + "id": 2310, "method": "session.queue.enqueue", "error": { "code": "invalid_idempotency_key", @@ -3685,7 +5096,7 @@ }, { "kind": "error", - "id": 1811, + "id": 2311, "method": "session.queue.enqueue", "error": { "code": "idempotency_conflict", @@ -3695,7 +5106,7 @@ }, { "kind": "error", - "id": 1812, + "id": 2312, "method": "session.queue.enqueue", "error": { "code": "blob_not_owned", @@ -3705,7 +5116,7 @@ }, { "kind": "error", - "id": 1813, + "id": 2313, "method": "session.queue.enqueue", "error": { "code": "blob_missing", @@ -3715,7 +5126,7 @@ }, { "kind": "error", - "id": 1814, + "id": 2314, "method": "session.queue.enqueue", "error": { "code": "blob_corrupt", @@ -3725,7 +5136,7 @@ }, { "kind": "error", - "id": 1815, + "id": 2315, "method": "session.queue.enqueue", "error": { "code": "content_too_large", @@ -3735,7 +5146,7 @@ }, { "kind": "error", - "id": 1901, + "id": 2401, "method": "session.queue.requeue", "error": { "code": "daemon_stopping", @@ -3745,7 +5156,7 @@ }, { "kind": "error", - "id": 1902, + "id": 2402, "method": "session.queue.requeue", "error": { "code": "bad_request", @@ -3755,7 +5166,7 @@ }, { "kind": "error", - "id": 1903, + "id": 2403, "method": "session.queue.requeue", "error": { "code": "not_initialized", @@ -3765,7 +5176,7 @@ }, { "kind": "error", - "id": 1904, + "id": 2404, "method": "session.queue.requeue", "error": { "code": "unsupported_capability", @@ -3775,7 +5186,7 @@ }, { "kind": "error", - "id": 1905, + "id": 2405, "method": "session.queue.requeue", "error": { "code": "rate_limited", @@ -3785,7 +5196,7 @@ }, { "kind": "error", - "id": 1906, + "id": 2406, "method": "session.queue.requeue", "error": { "code": "internal_error", @@ -3795,7 +5206,7 @@ }, { "kind": "error", - "id": 1907, + "id": 2407, "method": "session.queue.requeue", "error": { "code": "cancelled", @@ -3805,7 +5216,7 @@ }, { "kind": "error", - "id": 1908, + "id": 2408, "method": "session.queue.requeue", "error": { "code": "unknown_session", @@ -3815,7 +5226,7 @@ }, { "kind": "error", - "id": 1909, + "id": 2409, "method": "session.queue.requeue", "error": { "code": "event_migration_required", @@ -3825,7 +5236,7 @@ }, { "kind": "error", - "id": 1910, + "id": 2410, "method": "session.queue.requeue", "error": { "code": "unknown_queue_item", @@ -3835,7 +5246,7 @@ }, { "kind": "error", - "id": 1911, + "id": 2411, "method": "session.queue.requeue", "error": { "code": "queue_not_paused", @@ -3845,7 +5256,7 @@ }, { "kind": "error", - "id": 1912, + "id": 2412, "method": "session.queue.requeue", "error": { "code": "invalid_idempotency_key", @@ -3855,7 +5266,7 @@ }, { "kind": "error", - "id": 1913, + "id": 2413, "method": "session.queue.requeue", "error": { "code": "idempotency_conflict", @@ -3865,7 +5276,7 @@ }, { "kind": "error", - "id": 1914, + "id": 2414, "method": "session.queue.requeue", "error": { "code": "content_too_large", @@ -3875,7 +5286,7 @@ }, { "kind": "error", - "id": 2001, + "id": 2501, "method": "session.shell", "error": { "code": "daemon_stopping", @@ -3885,7 +5296,7 @@ }, { "kind": "error", - "id": 2002, + "id": 2502, "method": "session.shell", "error": { "code": "bad_request", @@ -3895,7 +5306,7 @@ }, { "kind": "error", - "id": 2003, + "id": 2503, "method": "session.shell", "error": { "code": "not_initialized", @@ -3905,7 +5316,7 @@ }, { "kind": "error", - "id": 2004, + "id": 2504, "method": "session.shell", "error": { "code": "unsupported_capability", @@ -3915,7 +5326,7 @@ }, { "kind": "error", - "id": 2005, + "id": 2505, "method": "session.shell", "error": { "code": "rate_limited", @@ -3925,7 +5336,7 @@ }, { "kind": "error", - "id": 2006, + "id": 2506, "method": "session.shell", "error": { "code": "internal_error", @@ -3935,7 +5346,7 @@ }, { "kind": "error", - "id": 2007, + "id": 2507, "method": "session.shell", "error": { "code": "cancelled", @@ -3945,7 +5356,7 @@ }, { "kind": "error", - "id": 2008, + "id": 2508, "method": "session.shell", "error": { "code": "unknown_session", @@ -3955,7 +5366,7 @@ }, { "kind": "error", - "id": 2009, + "id": 2509, "method": "session.shell", "error": { "code": "event_migration_required", @@ -3965,7 +5376,7 @@ }, { "kind": "error", - "id": 2010, + "id": 2510, "method": "session.shell", "error": { "code": "operation_active", @@ -3975,7 +5386,7 @@ }, { "kind": "error", - "id": 2011, + "id": 2511, "method": "session.shell", "error": { "code": "idempotency_conflict", @@ -3985,7 +5396,7 @@ }, { "kind": "error", - "id": 2012, + "id": 2512, "method": "session.shell", "error": { "code": "content_too_large", @@ -3995,7 +5406,7 @@ }, { "kind": "error", - "id": 2101, + "id": 2601, "method": "session.interrupt", "error": { "code": "daemon_stopping", @@ -4005,7 +5416,7 @@ }, { "kind": "error", - "id": 2102, + "id": 2602, "method": "session.interrupt", "error": { "code": "bad_request", @@ -4015,7 +5426,7 @@ }, { "kind": "error", - "id": 2103, + "id": 2603, "method": "session.interrupt", "error": { "code": "not_initialized", @@ -4025,7 +5436,7 @@ }, { "kind": "error", - "id": 2104, + "id": 2604, "method": "session.interrupt", "error": { "code": "unsupported_capability", @@ -4035,7 +5446,7 @@ }, { "kind": "error", - "id": 2105, + "id": 2605, "method": "session.interrupt", "error": { "code": "rate_limited", @@ -4045,7 +5456,7 @@ }, { "kind": "error", - "id": 2106, + "id": 2606, "method": "session.interrupt", "error": { "code": "internal_error", @@ -4055,7 +5466,7 @@ }, { "kind": "error", - "id": 2107, + "id": 2607, "method": "session.interrupt", "error": { "code": "cancelled", @@ -4065,187 +5476,341 @@ }, { "kind": "error", - "id": 2108, - "method": "session.interrupt", + "id": 2608, + "method": "session.interrupt", + "error": { + "code": "unknown_session", + "message": "Fixture session.interrupt error: unknown_session", + "retryable": false + } + }, + { + "kind": "error", + "id": 2609, + "method": "session.interrupt", + "error": { + "code": "event_migration_required", + "message": "Fixture session.interrupt error: event_migration_required", + "retryable": false + } + }, + { + "kind": "error", + "id": 2610, + "method": "session.interrupt", + "error": { + "code": "invalid_idempotency_key", + "message": "Fixture session.interrupt error: invalid_idempotency_key", + "retryable": false + } + }, + { + "kind": "error", + "id": 2611, + "method": "session.interrupt", + "error": { + "code": "idempotency_conflict", + "message": "Fixture session.interrupt error: idempotency_conflict", + "retryable": false + } + }, + { + "kind": "error", + "id": 2701, + "method": "session.reload", + "error": { + "code": "daemon_stopping", + "message": "Fixture session.reload error: daemon_stopping", + "retryable": false + } + }, + { + "kind": "error", + "id": 2702, + "method": "session.reload", + "error": { + "code": "bad_request", + "message": "Fixture session.reload error: bad_request", + "retryable": false + } + }, + { + "kind": "error", + "id": 2703, + "method": "session.reload", + "error": { + "code": "not_initialized", + "message": "Fixture session.reload error: not_initialized", + "retryable": false + } + }, + { + "kind": "error", + "id": 2704, + "method": "session.reload", + "error": { + "code": "unsupported_capability", + "message": "Fixture session.reload error: unsupported_capability", + "retryable": false + } + }, + { + "kind": "error", + "id": 2705, + "method": "session.reload", + "error": { + "code": "rate_limited", + "message": "Fixture session.reload error: rate_limited", + "retryable": true + } + }, + { + "kind": "error", + "id": 2706, + "method": "session.reload", + "error": { + "code": "internal_error", + "message": "Fixture session.reload error: internal_error", + "retryable": false + } + }, + { + "kind": "error", + "id": 2707, + "method": "session.reload", + "error": { + "code": "cancelled", + "message": "Fixture session.reload error: cancelled", + "retryable": false + } + }, + { + "kind": "error", + "id": 2708, + "method": "session.reload", "error": { "code": "unknown_session", - "message": "Fixture session.interrupt error: unknown_session", + "message": "Fixture session.reload error: unknown_session", "retryable": false } }, { "kind": "error", - "id": 2109, - "method": "session.interrupt", + "id": 2709, + "method": "session.reload", "error": { "code": "event_migration_required", - "message": "Fixture session.interrupt error: event_migration_required", + "message": "Fixture session.reload error: event_migration_required", "retryable": false } }, { "kind": "error", - "id": 2110, - "method": "session.interrupt", + "id": 2710, + "method": "session.reload", "error": { - "code": "invalid_idempotency_key", - "message": "Fixture session.interrupt error: invalid_idempotency_key", + "code": "corrupt_session", + "message": "Fixture session.reload error: corrupt_session", "retryable": false } }, { "kind": "error", - "id": 2111, - "method": "session.interrupt", + "id": 2711, + "method": "session.reload", "error": { - "code": "idempotency_conflict", - "message": "Fixture session.interrupt error: idempotency_conflict", + "code": "operation_active", + "message": "Fixture session.reload error: operation_active", "retryable": false } }, { "kind": "error", - "id": 2201, + "id": 2712, "method": "session.reload", "error": { - "code": "daemon_stopping", - "message": "Fixture session.reload error: daemon_stopping", + "code": "invalid_idempotency_key", + "message": "Fixture session.reload error: invalid_idempotency_key", "retryable": false } }, { "kind": "error", - "id": 2202, + "id": 2713, "method": "session.reload", "error": { - "code": "bad_request", - "message": "Fixture session.reload error: bad_request", + "code": "idempotency_conflict", + "message": "Fixture session.reload error: idempotency_conflict", "retryable": false } }, { "kind": "error", - "id": 2203, + "id": 2714, "method": "session.reload", "error": { - "code": "not_initialized", - "message": "Fixture session.reload error: not_initialized", + "code": "content_too_large", + "message": "Fixture session.reload error: content_too_large", "retryable": false } }, { "kind": "error", - "id": 2204, + "id": 2715, "method": "session.reload", "error": { - "code": "unsupported_capability", - "message": "Fixture session.reload error: unsupported_capability", - "retryable": false + "code": "provider_not_found", + "message": "Fixture session.reload error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2205, + "id": 2716, "method": "session.reload", "error": { - "code": "rate_limited", - "message": "Fixture session.reload error: rate_limited", - "retryable": true + "code": "provider_disabled", + "message": "Fixture session.reload error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2206, + "id": 2717, "method": "session.reload", "error": { - "code": "internal_error", - "message": "Fixture session.reload error: internal_error", - "retryable": false + "code": "model_not_found", + "message": "Fixture session.reload error: model_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2207, + "id": 2718, "method": "session.reload", "error": { - "code": "cancelled", - "message": "Fixture session.reload error: cancelled", - "retryable": false + "code": "model_unavailable", + "message": "Fixture session.reload error: model_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2208, + "id": 2719, "method": "session.reload", "error": { - "code": "unknown_session", - "message": "Fixture session.reload error: unknown_session", - "retryable": false + "code": "authentication_required", + "message": "Fixture session.reload error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2209, + "id": 2720, "method": "session.reload", "error": { - "code": "event_migration_required", - "message": "Fixture session.reload error: event_migration_required", - "retryable": false + "code": "authentication_failed", + "message": "Fixture session.reload error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2210, + "id": 2721, "method": "session.reload", "error": { - "code": "corrupt_session", - "message": "Fixture session.reload error: corrupt_session", - "retryable": false + "code": "entitlement_required", + "message": "Fixture session.reload error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2211, + "id": 2722, "method": "session.reload", "error": { - "code": "operation_active", - "message": "Fixture session.reload error: operation_active", - "retryable": false + "code": "entitlement_exhausted", + "message": "Fixture session.reload error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2212, + "id": 2723, "method": "session.reload", "error": { - "code": "invalid_idempotency_key", - "message": "Fixture session.reload error: invalid_idempotency_key", - "retryable": false + "code": "region_required", + "message": "Fixture session.reload error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2213, + "id": 2724, "method": "session.reload", "error": { - "code": "idempotency_conflict", - "message": "Fixture session.reload error: idempotency_conflict", - "retryable": false + "code": "region_unsupported", + "message": "Fixture session.reload error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2214, + "id": 2725, "method": "session.reload", "error": { - "code": "content_too_large", - "message": "Fixture session.reload error: content_too_large", - "retryable": false + "code": "provider_configuration_required", + "message": "Fixture session.reload error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } } }, { "kind": "error", - "id": 2301, + "id": 2801, "method": "session.configure", "error": { "code": "daemon_stopping", @@ -4255,7 +5820,7 @@ }, { "kind": "error", - "id": 2302, + "id": 2802, "method": "session.configure", "error": { "code": "bad_request", @@ -4265,7 +5830,7 @@ }, { "kind": "error", - "id": 2303, + "id": 2803, "method": "session.configure", "error": { "code": "not_initialized", @@ -4275,7 +5840,7 @@ }, { "kind": "error", - "id": 2304, + "id": 2804, "method": "session.configure", "error": { "code": "unsupported_capability", @@ -4285,7 +5850,7 @@ }, { "kind": "error", - "id": 2305, + "id": 2805, "method": "session.configure", "error": { "code": "rate_limited", @@ -4295,7 +5860,7 @@ }, { "kind": "error", - "id": 2306, + "id": 2806, "method": "session.configure", "error": { "code": "internal_error", @@ -4305,7 +5870,7 @@ }, { "kind": "error", - "id": 2307, + "id": 2807, "method": "session.configure", "error": { "code": "cancelled", @@ -4315,7 +5880,7 @@ }, { "kind": "error", - "id": 2308, + "id": 2808, "method": "session.configure", "error": { "code": "unknown_session", @@ -4325,7 +5890,7 @@ }, { "kind": "error", - "id": 2309, + "id": 2809, "method": "session.configure", "error": { "code": "event_migration_required", @@ -4335,7 +5900,7 @@ }, { "kind": "error", - "id": 2310, + "id": 2810, "method": "session.configure", "error": { "code": "corrupt_session", @@ -4345,7 +5910,7 @@ }, { "kind": "error", - "id": 2311, + "id": 2811, "method": "session.configure", "error": { "code": "operation_active", @@ -4355,7 +5920,7 @@ }, { "kind": "error", - "id": 2312, + "id": 2812, "method": "session.configure", "error": { "code": "invalid_idempotency_key", @@ -4365,7 +5930,7 @@ }, { "kind": "error", - "id": 2313, + "id": 2813, "method": "session.configure", "error": { "code": "idempotency_conflict", @@ -4375,7 +5940,7 @@ }, { "kind": "error", - "id": 2314, + "id": 2814, "method": "session.configure", "error": { "code": "content_too_large", @@ -4385,7 +5950,161 @@ }, { "kind": "error", - "id": 2401, + "id": 2815, + "method": "session.configure", + "error": { + "code": "provider_not_found", + "message": "Fixture session.configure error: provider_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2816, + "method": "session.configure", + "error": { + "code": "provider_disabled", + "message": "Fixture session.configure error: provider_disabled", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2817, + "method": "session.configure", + "error": { + "code": "model_not_found", + "message": "Fixture session.configure error: model_not_found", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2818, + "method": "session.configure", + "error": { + "code": "model_unavailable", + "message": "Fixture session.configure error: model_unavailable", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2819, + "method": "session.configure", + "error": { + "code": "authentication_required", + "message": "Fixture session.configure error: authentication_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2820, + "method": "session.configure", + "error": { + "code": "authentication_failed", + "message": "Fixture session.configure error: authentication_failed", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2821, + "method": "session.configure", + "error": { + "code": "entitlement_required", + "message": "Fixture session.configure error: entitlement_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2822, + "method": "session.configure", + "error": { + "code": "entitlement_exhausted", + "message": "Fixture session.configure error: entitlement_exhausted", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2823, + "method": "session.configure", + "error": { + "code": "region_required", + "message": "Fixture session.configure error: region_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2824, + "method": "session.configure", + "error": { + "code": "region_unsupported", + "message": "Fixture session.configure error: region_unsupported", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2825, + "method": "session.configure", + "error": { + "code": "provider_configuration_required", + "message": "Fixture session.configure error: provider_configuration_required", + "retryable": false, + "details": { + "category": "provider", + "action": "configure_provider" + } + } + }, + { + "kind": "error", + "id": 2901, "method": "session.interaction.respond", "error": { "code": "daemon_stopping", @@ -4395,7 +6114,7 @@ }, { "kind": "error", - "id": 2402, + "id": 2902, "method": "session.interaction.respond", "error": { "code": "bad_request", @@ -4405,7 +6124,7 @@ }, { "kind": "error", - "id": 2403, + "id": 2903, "method": "session.interaction.respond", "error": { "code": "not_initialized", @@ -4415,7 +6134,7 @@ }, { "kind": "error", - "id": 2404, + "id": 2904, "method": "session.interaction.respond", "error": { "code": "unsupported_capability", @@ -4425,7 +6144,7 @@ }, { "kind": "error", - "id": 2405, + "id": 2905, "method": "session.interaction.respond", "error": { "code": "rate_limited", @@ -4435,7 +6154,7 @@ }, { "kind": "error", - "id": 2406, + "id": 2906, "method": "session.interaction.respond", "error": { "code": "internal_error", @@ -4445,7 +6164,7 @@ }, { "kind": "error", - "id": 2407, + "id": 2907, "method": "session.interaction.respond", "error": { "code": "cancelled", @@ -4455,7 +6174,7 @@ }, { "kind": "error", - "id": 2408, + "id": 2908, "method": "session.interaction.respond", "error": { "code": "unknown_session", @@ -4465,7 +6184,7 @@ }, { "kind": "error", - "id": 2409, + "id": 2909, "method": "session.interaction.respond", "error": { "code": "event_migration_required", @@ -4475,7 +6194,7 @@ }, { "kind": "error", - "id": 2410, + "id": 2910, "method": "session.interaction.respond", "error": { "code": "unknown_interaction", @@ -4485,7 +6204,7 @@ }, { "kind": "error", - "id": 2411, + "id": 2911, "method": "session.interaction.respond", "error": { "code": "interaction_already_resolved", @@ -4495,7 +6214,7 @@ }, { "kind": "error", - "id": 2412, + "id": 2912, "method": "session.interaction.respond", "error": { "code": "invalid_idempotency_key", @@ -4505,7 +6224,7 @@ }, { "kind": "error", - "id": 2413, + "id": 2913, "method": "session.interaction.respond", "error": { "code": "idempotency_conflict", @@ -4515,7 +6234,7 @@ }, { "kind": "error", - "id": 2414, + "id": 2914, "method": "session.interaction.respond", "error": { "code": "content_too_large", @@ -4525,7 +6244,7 @@ }, { "kind": "error", - "id": 2501, + "id": 3001, "method": "session.subscribe", "error": { "code": "daemon_stopping", @@ -4535,7 +6254,7 @@ }, { "kind": "error", - "id": 2502, + "id": 3002, "method": "session.subscribe", "error": { "code": "bad_request", @@ -4545,7 +6264,7 @@ }, { "kind": "error", - "id": 2503, + "id": 3003, "method": "session.subscribe", "error": { "code": "not_initialized", @@ -4555,7 +6274,7 @@ }, { "kind": "error", - "id": 2504, + "id": 3004, "method": "session.subscribe", "error": { "code": "unsupported_capability", @@ -4565,7 +6284,7 @@ }, { "kind": "error", - "id": 2505, + "id": 3005, "method": "session.subscribe", "error": { "code": "rate_limited", @@ -4575,7 +6294,7 @@ }, { "kind": "error", - "id": 2506, + "id": 3006, "method": "session.subscribe", "error": { "code": "internal_error", @@ -4585,7 +6304,7 @@ }, { "kind": "error", - "id": 2507, + "id": 3007, "method": "session.subscribe", "error": { "code": "cancelled", @@ -4595,7 +6314,7 @@ }, { "kind": "error", - "id": 2508, + "id": 3008, "method": "session.subscribe", "error": { "code": "unknown_session", @@ -4605,7 +6324,7 @@ }, { "kind": "error", - "id": 2509, + "id": 3009, "method": "session.subscribe", "error": { "code": "event_migration_required", @@ -4615,7 +6334,7 @@ }, { "kind": "error", - "id": 2510, + "id": 3010, "method": "session.subscribe", "error": { "code": "snapshot_required", @@ -4625,7 +6344,7 @@ }, { "kind": "error", - "id": 2601, + "id": 3101, "method": "session.workspace.list", "error": { "code": "daemon_stopping", @@ -4635,7 +6354,7 @@ }, { "kind": "error", - "id": 2602, + "id": 3102, "method": "session.workspace.list", "error": { "code": "bad_request", @@ -4645,7 +6364,7 @@ }, { "kind": "error", - "id": 2603, + "id": 3103, "method": "session.workspace.list", "error": { "code": "not_initialized", @@ -4655,7 +6374,7 @@ }, { "kind": "error", - "id": 2604, + "id": 3104, "method": "session.workspace.list", "error": { "code": "unsupported_capability", @@ -4665,7 +6384,7 @@ }, { "kind": "error", - "id": 2605, + "id": 3105, "method": "session.workspace.list", "error": { "code": "rate_limited", @@ -4675,7 +6394,7 @@ }, { "kind": "error", - "id": 2606, + "id": 3106, "method": "session.workspace.list", "error": { "code": "internal_error", @@ -4685,7 +6404,7 @@ }, { "kind": "error", - "id": 2607, + "id": 3107, "method": "session.workspace.list", "error": { "code": "cancelled", @@ -4695,7 +6414,7 @@ }, { "kind": "error", - "id": 2608, + "id": 3108, "method": "session.workspace.list", "error": { "code": "unknown_session", @@ -4705,7 +6424,7 @@ }, { "kind": "error", - "id": 2609, + "id": 3109, "method": "session.workspace.list", "error": { "code": "event_migration_required", @@ -4715,7 +6434,7 @@ }, { "kind": "error", - "id": 2610, + "id": 3110, "method": "session.workspace.list", "error": { "code": "workspace_unavailable", @@ -4725,7 +6444,7 @@ }, { "kind": "error", - "id": 2611, + "id": 3111, "method": "session.workspace.list", "error": { "code": "workspace_changed", @@ -4735,7 +6454,7 @@ }, { "kind": "error", - "id": 2612, + "id": 3112, "method": "session.workspace.list", "error": { "code": "invalid_path", @@ -4745,7 +6464,7 @@ }, { "kind": "error", - "id": 2613, + "id": 3113, "method": "session.workspace.list", "error": { "code": "path_denied", @@ -4755,7 +6474,7 @@ }, { "kind": "error", - "id": 2614, + "id": 3114, "method": "session.workspace.list", "error": { "code": "symlink_escape", @@ -4765,7 +6484,7 @@ }, { "kind": "error", - "id": 2615, + "id": 3115, "method": "session.workspace.list", "error": { "code": "not_found", @@ -4775,7 +6494,7 @@ }, { "kind": "error", - "id": 2616, + "id": 3116, "method": "session.workspace.list", "error": { "code": "unsupported_file_type", @@ -4785,7 +6504,7 @@ }, { "kind": "error", - "id": 2617, + "id": 3117, "method": "session.workspace.list", "error": { "code": "unsupported_filename_encoding", @@ -4795,7 +6514,7 @@ }, { "kind": "error", - "id": 2701, + "id": 3201, "method": "session.workspace.read", "error": { "code": "daemon_stopping", @@ -4805,7 +6524,7 @@ }, { "kind": "error", - "id": 2702, + "id": 3202, "method": "session.workspace.read", "error": { "code": "bad_request", @@ -4815,7 +6534,7 @@ }, { "kind": "error", - "id": 2703, + "id": 3203, "method": "session.workspace.read", "error": { "code": "not_initialized", @@ -4825,7 +6544,7 @@ }, { "kind": "error", - "id": 2704, + "id": 3204, "method": "session.workspace.read", "error": { "code": "unsupported_capability", @@ -4835,7 +6554,7 @@ }, { "kind": "error", - "id": 2705, + "id": 3205, "method": "session.workspace.read", "error": { "code": "rate_limited", @@ -4845,7 +6564,7 @@ }, { "kind": "error", - "id": 2706, + "id": 3206, "method": "session.workspace.read", "error": { "code": "internal_error", @@ -4855,7 +6574,7 @@ }, { "kind": "error", - "id": 2707, + "id": 3207, "method": "session.workspace.read", "error": { "code": "cancelled", @@ -4865,7 +6584,7 @@ }, { "kind": "error", - "id": 2708, + "id": 3208, "method": "session.workspace.read", "error": { "code": "unknown_session", @@ -4875,7 +6594,7 @@ }, { "kind": "error", - "id": 2709, + "id": 3209, "method": "session.workspace.read", "error": { "code": "event_migration_required", @@ -4885,7 +6604,7 @@ }, { "kind": "error", - "id": 2710, + "id": 3210, "method": "session.workspace.read", "error": { "code": "workspace_unavailable", @@ -4895,7 +6614,7 @@ }, { "kind": "error", - "id": 2711, + "id": 3211, "method": "session.workspace.read", "error": { "code": "workspace_changed", @@ -4905,7 +6624,7 @@ }, { "kind": "error", - "id": 2712, + "id": 3212, "method": "session.workspace.read", "error": { "code": "invalid_path", @@ -4915,7 +6634,7 @@ }, { "kind": "error", - "id": 2713, + "id": 3213, "method": "session.workspace.read", "error": { "code": "path_denied", @@ -4925,7 +6644,7 @@ }, { "kind": "error", - "id": 2714, + "id": 3214, "method": "session.workspace.read", "error": { "code": "symlink_escape", @@ -4935,7 +6654,7 @@ }, { "kind": "error", - "id": 2715, + "id": 3215, "method": "session.workspace.read", "error": { "code": "not_found", @@ -4945,7 +6664,7 @@ }, { "kind": "error", - "id": 2716, + "id": 3216, "method": "session.workspace.read", "error": { "code": "not_a_file", @@ -4955,7 +6674,7 @@ }, { "kind": "error", - "id": 2717, + "id": 3217, "method": "session.workspace.read", "error": { "code": "unsupported_file_type", @@ -4965,7 +6684,7 @@ }, { "kind": "error", - "id": 2718, + "id": 3218, "method": "session.workspace.read", "error": { "code": "binary_file", @@ -4975,7 +6694,7 @@ }, { "kind": "error", - "id": 2719, + "id": 3219, "method": "session.workspace.read", "error": { "code": "invalid_encoding", @@ -4985,7 +6704,7 @@ }, { "kind": "error", - "id": 2720, + "id": 3220, "method": "session.workspace.read", "error": { "code": "content_too_large", @@ -4995,7 +6714,7 @@ }, { "kind": "error", - "id": 2801, + "id": 3301, "method": "session.workspace.status", "error": { "code": "daemon_stopping", @@ -5005,7 +6724,7 @@ }, { "kind": "error", - "id": 2802, + "id": 3302, "method": "session.workspace.status", "error": { "code": "bad_request", @@ -5015,7 +6734,7 @@ }, { "kind": "error", - "id": 2803, + "id": 3303, "method": "session.workspace.status", "error": { "code": "not_initialized", @@ -5025,7 +6744,7 @@ }, { "kind": "error", - "id": 2804, + "id": 3304, "method": "session.workspace.status", "error": { "code": "unsupported_capability", @@ -5035,7 +6754,7 @@ }, { "kind": "error", - "id": 2805, + "id": 3305, "method": "session.workspace.status", "error": { "code": "rate_limited", @@ -5045,7 +6764,7 @@ }, { "kind": "error", - "id": 2806, + "id": 3306, "method": "session.workspace.status", "error": { "code": "internal_error", @@ -5055,7 +6774,7 @@ }, { "kind": "error", - "id": 2807, + "id": 3307, "method": "session.workspace.status", "error": { "code": "cancelled", @@ -5065,7 +6784,7 @@ }, { "kind": "error", - "id": 2808, + "id": 3308, "method": "session.workspace.status", "error": { "code": "unknown_session", @@ -5075,7 +6794,7 @@ }, { "kind": "error", - "id": 2809, + "id": 3309, "method": "session.workspace.status", "error": { "code": "event_migration_required", @@ -5085,7 +6804,7 @@ }, { "kind": "error", - "id": 2810, + "id": 3310, "method": "session.workspace.status", "error": { "code": "workspace_unavailable", @@ -5095,7 +6814,7 @@ }, { "kind": "error", - "id": 2811, + "id": 3311, "method": "session.workspace.status", "error": { "code": "workspace_changed", @@ -5105,7 +6824,7 @@ }, { "kind": "error", - "id": 2812, + "id": 3312, "method": "session.workspace.status", "error": { "code": "not_git_repository", @@ -5115,7 +6834,7 @@ }, { "kind": "error", - "id": 2813, + "id": 3313, "method": "session.workspace.status", "error": { "code": "git_unavailable", @@ -5125,7 +6844,7 @@ }, { "kind": "error", - "id": 2814, + "id": 3314, "method": "session.workspace.status", "error": { "code": "git_timeout", @@ -5135,7 +6854,7 @@ }, { "kind": "error", - "id": 2815, + "id": 3315, "method": "session.workspace.status", "error": { "code": "git_output_too_large", @@ -5145,7 +6864,7 @@ }, { "kind": "error", - "id": 2816, + "id": 3316, "method": "session.workspace.status", "error": { "code": "unsupported_git_state", @@ -5155,7 +6874,7 @@ }, { "kind": "error", - "id": 2817, + "id": 3317, "method": "session.workspace.status", "error": { "code": "unsupported_filename_encoding", @@ -5165,7 +6884,7 @@ }, { "kind": "error", - "id": 2818, + "id": 3318, "method": "session.workspace.status", "error": { "code": "checkpoint_unavailable", @@ -5175,7 +6894,7 @@ }, { "kind": "error", - "id": 2819, + "id": 3319, "method": "session.workspace.status", "error": { "code": "checkpoint_too_large", @@ -5185,7 +6904,7 @@ }, { "kind": "error", - "id": 2820, + "id": 3320, "method": "session.workspace.status", "error": { "code": "checkpoint_corrupt", @@ -5195,7 +6914,7 @@ }, { "kind": "error", - "id": 2821, + "id": 3321, "method": "session.workspace.status", "error": { "code": "path_denied", @@ -5205,7 +6924,7 @@ }, { "kind": "error", - "id": 2901, + "id": 3401, "method": "session.workspace.diff", "error": { "code": "daemon_stopping", @@ -5215,7 +6934,7 @@ }, { "kind": "error", - "id": 2902, + "id": 3402, "method": "session.workspace.diff", "error": { "code": "bad_request", @@ -5225,7 +6944,7 @@ }, { "kind": "error", - "id": 2903, + "id": 3403, "method": "session.workspace.diff", "error": { "code": "not_initialized", @@ -5235,7 +6954,7 @@ }, { "kind": "error", - "id": 2904, + "id": 3404, "method": "session.workspace.diff", "error": { "code": "unsupported_capability", @@ -5245,7 +6964,7 @@ }, { "kind": "error", - "id": 2905, + "id": 3405, "method": "session.workspace.diff", "error": { "code": "rate_limited", @@ -5255,7 +6974,7 @@ }, { "kind": "error", - "id": 2906, + "id": 3406, "method": "session.workspace.diff", "error": { "code": "internal_error", @@ -5265,7 +6984,7 @@ }, { "kind": "error", - "id": 2907, + "id": 3407, "method": "session.workspace.diff", "error": { "code": "cancelled", @@ -5275,7 +6994,7 @@ }, { "kind": "error", - "id": 2908, + "id": 3408, "method": "session.workspace.diff", "error": { "code": "unknown_session", @@ -5285,7 +7004,7 @@ }, { "kind": "error", - "id": 2909, + "id": 3409, "method": "session.workspace.diff", "error": { "code": "event_migration_required", @@ -5295,7 +7014,7 @@ }, { "kind": "error", - "id": 2910, + "id": 3410, "method": "session.workspace.diff", "error": { "code": "workspace_unavailable", @@ -5305,7 +7024,7 @@ }, { "kind": "error", - "id": 2911, + "id": 3411, "method": "session.workspace.diff", "error": { "code": "workspace_changed", @@ -5315,7 +7034,7 @@ }, { "kind": "error", - "id": 2912, + "id": 3412, "method": "session.workspace.diff", "error": { "code": "not_git_repository", @@ -5325,7 +7044,7 @@ }, { "kind": "error", - "id": 2913, + "id": 3413, "method": "session.workspace.diff", "error": { "code": "git_unavailable", @@ -5335,7 +7054,7 @@ }, { "kind": "error", - "id": 2914, + "id": 3414, "method": "session.workspace.diff", "error": { "code": "git_timeout", @@ -5345,7 +7064,7 @@ }, { "kind": "error", - "id": 2915, + "id": 3415, "method": "session.workspace.diff", "error": { "code": "git_output_too_large", @@ -5355,7 +7074,7 @@ }, { "kind": "error", - "id": 2916, + "id": 3416, "method": "session.workspace.diff", "error": { "code": "unsupported_git_state", @@ -5365,7 +7084,7 @@ }, { "kind": "error", - "id": 2917, + "id": 3417, "method": "session.workspace.diff", "error": { "code": "unsupported_filename_encoding", @@ -5375,7 +7094,7 @@ }, { "kind": "error", - "id": 2918, + "id": 3418, "method": "session.workspace.diff", "error": { "code": "checkpoint_unavailable", @@ -5385,7 +7104,7 @@ }, { "kind": "error", - "id": 2919, + "id": 3419, "method": "session.workspace.diff", "error": { "code": "checkpoint_too_large", @@ -5395,7 +7114,7 @@ }, { "kind": "error", - "id": 2920, + "id": 3420, "method": "session.workspace.diff", "error": { "code": "checkpoint_corrupt", @@ -5405,7 +7124,7 @@ }, { "kind": "error", - "id": 2921, + "id": 3421, "method": "session.workspace.diff", "error": { "code": "path_denied", @@ -5415,7 +7134,7 @@ }, { "kind": "error", - "id": 2922, + "id": 3422, "method": "session.workspace.diff", "error": { "code": "repository_changed", @@ -5425,7 +7144,7 @@ }, { "kind": "error", - "id": 3001, + "id": 3501, "method": "session.workspace.checkpoint", "error": { "code": "daemon_stopping", @@ -5435,7 +7154,7 @@ }, { "kind": "error", - "id": 3002, + "id": 3502, "method": "session.workspace.checkpoint", "error": { "code": "bad_request", @@ -5445,7 +7164,7 @@ }, { "kind": "error", - "id": 3003, + "id": 3503, "method": "session.workspace.checkpoint", "error": { "code": "not_initialized", @@ -5455,7 +7174,7 @@ }, { "kind": "error", - "id": 3004, + "id": 3504, "method": "session.workspace.checkpoint", "error": { "code": "unsupported_capability", @@ -5465,7 +7184,7 @@ }, { "kind": "error", - "id": 3005, + "id": 3505, "method": "session.workspace.checkpoint", "error": { "code": "rate_limited", @@ -5475,7 +7194,7 @@ }, { "kind": "error", - "id": 3006, + "id": 3506, "method": "session.workspace.checkpoint", "error": { "code": "internal_error", @@ -5485,7 +7204,7 @@ }, { "kind": "error", - "id": 3007, + "id": 3507, "method": "session.workspace.checkpoint", "error": { "code": "cancelled", @@ -5495,7 +7214,7 @@ }, { "kind": "error", - "id": 3008, + "id": 3508, "method": "session.workspace.checkpoint", "error": { "code": "unknown_session", @@ -5505,7 +7224,7 @@ }, { "kind": "error", - "id": 3009, + "id": 3509, "method": "session.workspace.checkpoint", "error": { "code": "event_migration_required", @@ -5515,7 +7234,7 @@ }, { "kind": "error", - "id": 3010, + "id": 3510, "method": "session.workspace.checkpoint", "error": { "code": "operation_active", @@ -5525,7 +7244,7 @@ }, { "kind": "error", - "id": 3011, + "id": 3511, "method": "session.workspace.checkpoint", "error": { "code": "not_git_repository", @@ -5535,7 +7254,7 @@ }, { "kind": "error", - "id": 3012, + "id": 3512, "method": "session.workspace.checkpoint", "error": { "code": "git_unavailable", @@ -5545,7 +7264,7 @@ }, { "kind": "error", - "id": 3013, + "id": 3513, "method": "session.workspace.checkpoint", "error": { "code": "git_timeout", @@ -5555,7 +7274,7 @@ }, { "kind": "error", - "id": 3014, + "id": 3514, "method": "session.workspace.checkpoint", "error": { "code": "git_output_too_large", @@ -5565,7 +7284,7 @@ }, { "kind": "error", - "id": 3015, + "id": 3515, "method": "session.workspace.checkpoint", "error": { "code": "unsupported_git_state", @@ -5575,7 +7294,7 @@ }, { "kind": "error", - "id": 3016, + "id": 3516, "method": "session.workspace.checkpoint", "error": { "code": "unsupported_filename_encoding", @@ -5585,7 +7304,7 @@ }, { "kind": "error", - "id": 3017, + "id": 3517, "method": "session.workspace.checkpoint", "error": { "code": "checkpoint_unavailable", @@ -5595,7 +7314,7 @@ }, { "kind": "error", - "id": 3018, + "id": 3518, "method": "session.workspace.checkpoint", "error": { "code": "checkpoint_too_large", @@ -5605,7 +7324,7 @@ }, { "kind": "error", - "id": 3019, + "id": 3519, "method": "session.workspace.checkpoint", "error": { "code": "checkpoint_corrupt", @@ -5615,7 +7334,7 @@ }, { "kind": "error", - "id": 3101, + "id": 3601, "method": "session.blob.start", "error": { "code": "daemon_stopping", @@ -5625,7 +7344,7 @@ }, { "kind": "error", - "id": 3102, + "id": 3602, "method": "session.blob.start", "error": { "code": "bad_request", @@ -5635,7 +7354,7 @@ }, { "kind": "error", - "id": 3103, + "id": 3603, "method": "session.blob.start", "error": { "code": "not_initialized", @@ -5645,7 +7364,7 @@ }, { "kind": "error", - "id": 3104, + "id": 3604, "method": "session.blob.start", "error": { "code": "unsupported_capability", @@ -5655,7 +7374,7 @@ }, { "kind": "error", - "id": 3105, + "id": 3605, "method": "session.blob.start", "error": { "code": "rate_limited", @@ -5665,7 +7384,7 @@ }, { "kind": "error", - "id": 3106, + "id": 3606, "method": "session.blob.start", "error": { "code": "internal_error", @@ -5675,7 +7394,7 @@ }, { "kind": "error", - "id": 3107, + "id": 3607, "method": "session.blob.start", "error": { "code": "cancelled", @@ -5685,7 +7404,7 @@ }, { "kind": "error", - "id": 3108, + "id": 3608, "method": "session.blob.start", "error": { "code": "unknown_session", @@ -5695,7 +7414,7 @@ }, { "kind": "error", - "id": 3109, + "id": 3609, "method": "session.blob.start", "error": { "code": "event_migration_required", @@ -5705,7 +7424,7 @@ }, { "kind": "error", - "id": 3110, + "id": 3610, "method": "session.blob.start", "error": { "code": "invalid_media_type", @@ -5715,7 +7434,7 @@ }, { "kind": "error", - "id": 3111, + "id": 3611, "method": "session.blob.start", "error": { "code": "invalid_blob_name", @@ -5725,7 +7444,7 @@ }, { "kind": "error", - "id": 3112, + "id": 3612, "method": "session.blob.start", "error": { "code": "blob_too_large", @@ -5735,7 +7454,7 @@ }, { "kind": "error", - "id": 3113, + "id": 3613, "method": "session.blob.start", "error": { "code": "too_many_uploads", @@ -5745,7 +7464,7 @@ }, { "kind": "error", - "id": 3201, + "id": 3701, "method": "session.blob.chunk", "error": { "code": "daemon_stopping", @@ -5755,7 +7474,7 @@ }, { "kind": "error", - "id": 3202, + "id": 3702, "method": "session.blob.chunk", "error": { "code": "bad_request", @@ -5765,7 +7484,7 @@ }, { "kind": "error", - "id": 3203, + "id": 3703, "method": "session.blob.chunk", "error": { "code": "not_initialized", @@ -5775,7 +7494,7 @@ }, { "kind": "error", - "id": 3204, + "id": 3704, "method": "session.blob.chunk", "error": { "code": "unsupported_capability", @@ -5785,7 +7504,7 @@ }, { "kind": "error", - "id": 3205, + "id": 3705, "method": "session.blob.chunk", "error": { "code": "rate_limited", @@ -5795,7 +7514,7 @@ }, { "kind": "error", - "id": 3206, + "id": 3706, "method": "session.blob.chunk", "error": { "code": "internal_error", @@ -5805,7 +7524,7 @@ }, { "kind": "error", - "id": 3207, + "id": 3707, "method": "session.blob.chunk", "error": { "code": "cancelled", @@ -5815,7 +7534,7 @@ }, { "kind": "error", - "id": 3208, + "id": 3708, "method": "session.blob.chunk", "error": { "code": "unknown_session", @@ -5825,7 +7544,7 @@ }, { "kind": "error", - "id": 3209, + "id": 3709, "method": "session.blob.chunk", "error": { "code": "event_migration_required", @@ -5835,7 +7554,7 @@ }, { "kind": "error", - "id": 3210, + "id": 3710, "method": "session.blob.chunk", "error": { "code": "unknown_blob_upload", @@ -5845,7 +7564,7 @@ }, { "kind": "error", - "id": 3211, + "id": 3711, "method": "session.blob.chunk", "error": { "code": "invalid_blob_chunk", @@ -5855,7 +7574,7 @@ }, { "kind": "error", - "id": 3212, + "id": 3712, "method": "session.blob.chunk", "error": { "code": "blob_offset_mismatch", @@ -5865,7 +7584,7 @@ }, { "kind": "error", - "id": 3213, + "id": 3713, "method": "session.blob.chunk", "error": { "code": "blob_size_mismatch", @@ -5875,7 +7594,7 @@ }, { "kind": "error", - "id": 3214, + "id": 3714, "method": "session.blob.chunk", "error": { "code": "blob_write_failed", @@ -5885,7 +7604,7 @@ }, { "kind": "error", - "id": 3301, + "id": 3801, "method": "session.blob.commit", "error": { "code": "daemon_stopping", @@ -5895,7 +7614,7 @@ }, { "kind": "error", - "id": 3302, + "id": 3802, "method": "session.blob.commit", "error": { "code": "bad_request", @@ -5905,7 +7624,7 @@ }, { "kind": "error", - "id": 3303, + "id": 3803, "method": "session.blob.commit", "error": { "code": "not_initialized", @@ -5915,7 +7634,7 @@ }, { "kind": "error", - "id": 3304, + "id": 3804, "method": "session.blob.commit", "error": { "code": "unsupported_capability", @@ -5925,7 +7644,7 @@ }, { "kind": "error", - "id": 3305, + "id": 3805, "method": "session.blob.commit", "error": { "code": "rate_limited", @@ -5935,7 +7654,7 @@ }, { "kind": "error", - "id": 3306, + "id": 3806, "method": "session.blob.commit", "error": { "code": "internal_error", @@ -5945,7 +7664,7 @@ }, { "kind": "error", - "id": 3307, + "id": 3807, "method": "session.blob.commit", "error": { "code": "cancelled", @@ -5955,7 +7674,7 @@ }, { "kind": "error", - "id": 3308, + "id": 3808, "method": "session.blob.commit", "error": { "code": "unknown_session", @@ -5965,7 +7684,7 @@ }, { "kind": "error", - "id": 3309, + "id": 3809, "method": "session.blob.commit", "error": { "code": "event_migration_required", @@ -5975,7 +7694,7 @@ }, { "kind": "error", - "id": 3310, + "id": 3810, "method": "session.blob.commit", "error": { "code": "unknown_blob_upload", @@ -5985,7 +7704,7 @@ }, { "kind": "error", - "id": 3311, + "id": 3811, "method": "session.blob.commit", "error": { "code": "blob_size_mismatch", @@ -5995,7 +7714,7 @@ }, { "kind": "error", - "id": 3312, + "id": 3812, "method": "session.blob.commit", "error": { "code": "invalid_image", @@ -6005,7 +7724,7 @@ }, { "kind": "error", - "id": 3313, + "id": 3813, "method": "session.blob.commit", "error": { "code": "blob_corrupt", @@ -6015,7 +7734,7 @@ }, { "kind": "error", - "id": 3401, + "id": 3901, "method": "session.blob.abort", "error": { "code": "daemon_stopping", @@ -6025,7 +7744,7 @@ }, { "kind": "error", - "id": 3402, + "id": 3902, "method": "session.blob.abort", "error": { "code": "bad_request", @@ -6035,7 +7754,7 @@ }, { "kind": "error", - "id": 3403, + "id": 3903, "method": "session.blob.abort", "error": { "code": "not_initialized", @@ -6045,7 +7764,7 @@ }, { "kind": "error", - "id": 3404, + "id": 3904, "method": "session.blob.abort", "error": { "code": "unsupported_capability", @@ -6055,7 +7774,7 @@ }, { "kind": "error", - "id": 3405, + "id": 3905, "method": "session.blob.abort", "error": { "code": "rate_limited", @@ -6065,7 +7784,7 @@ }, { "kind": "error", - "id": 3406, + "id": 3906, "method": "session.blob.abort", "error": { "code": "internal_error", @@ -6075,7 +7794,7 @@ }, { "kind": "error", - "id": 3407, + "id": 3907, "method": "session.blob.abort", "error": { "code": "cancelled", @@ -6085,7 +7804,7 @@ }, { "kind": "error", - "id": 3408, + "id": 3908, "method": "session.blob.abort", "error": { "code": "unknown_session", @@ -6095,7 +7814,7 @@ }, { "kind": "error", - "id": 3409, + "id": 3909, "method": "session.blob.abort", "error": { "code": "event_migration_required", @@ -6105,7 +7824,7 @@ }, { "kind": "error", - "id": 3410, + "id": 3910, "method": "session.blob.abort", "error": { "code": "unknown_blob_upload", @@ -6115,7 +7834,7 @@ }, { "kind": "error", - "id": 3501, + "id": 4001, "method": "session.blob.read", "error": { "code": "daemon_stopping", @@ -6125,7 +7844,7 @@ }, { "kind": "error", - "id": 3502, + "id": 4002, "method": "session.blob.read", "error": { "code": "bad_request", @@ -6135,7 +7854,7 @@ }, { "kind": "error", - "id": 3503, + "id": 4003, "method": "session.blob.read", "error": { "code": "not_initialized", @@ -6145,7 +7864,7 @@ }, { "kind": "error", - "id": 3504, + "id": 4004, "method": "session.blob.read", "error": { "code": "unsupported_capability", @@ -6155,7 +7874,7 @@ }, { "kind": "error", - "id": 3505, + "id": 4005, "method": "session.blob.read", "error": { "code": "rate_limited", @@ -6165,7 +7884,7 @@ }, { "kind": "error", - "id": 3506, + "id": 4006, "method": "session.blob.read", "error": { "code": "internal_error", @@ -6175,7 +7894,7 @@ }, { "kind": "error", - "id": 3507, + "id": 4007, "method": "session.blob.read", "error": { "code": "cancelled", @@ -6185,7 +7904,7 @@ }, { "kind": "error", - "id": 3508, + "id": 4008, "method": "session.blob.read", "error": { "code": "unknown_session", @@ -6195,7 +7914,7 @@ }, { "kind": "error", - "id": 3509, + "id": 4009, "method": "session.blob.read", "error": { "code": "event_migration_required", @@ -6205,7 +7924,7 @@ }, { "kind": "error", - "id": 3510, + "id": 4010, "method": "session.blob.read", "error": { "code": "blob_not_owned", @@ -6215,7 +7934,7 @@ }, { "kind": "error", - "id": 3511, + "id": 4011, "method": "session.blob.read", "error": { "code": "blob_missing", @@ -6225,7 +7944,7 @@ }, { "kind": "error", - "id": 3512, + "id": 4012, "method": "session.blob.read", "error": { "code": "blob_corrupt", @@ -6235,7 +7954,7 @@ }, { "kind": "error", - "id": 3513, + "id": 4013, "method": "session.blob.read", "error": { "code": "invalid_blob_range", @@ -6245,7 +7964,7 @@ }, { "kind": "error", - "id": 3514, + "id": 4014, "method": "session.blob.read", "error": { "code": "blob_read_failed", @@ -6255,7 +7974,7 @@ }, { "kind": "error", - "id": 3601, + "id": 4101, "method": "session.dispose", "error": { "code": "daemon_stopping", @@ -6265,7 +7984,7 @@ }, { "kind": "error", - "id": 3602, + "id": 4102, "method": "session.dispose", "error": { "code": "bad_request", @@ -6275,7 +7994,7 @@ }, { "kind": "error", - "id": 3603, + "id": 4103, "method": "session.dispose", "error": { "code": "not_initialized", @@ -6285,7 +8004,7 @@ }, { "kind": "error", - "id": 3604, + "id": 4104, "method": "session.dispose", "error": { "code": "unsupported_capability", @@ -6295,7 +8014,7 @@ }, { "kind": "error", - "id": 3605, + "id": 4105, "method": "session.dispose", "error": { "code": "rate_limited", @@ -6305,7 +8024,7 @@ }, { "kind": "error", - "id": 3606, + "id": 4106, "method": "session.dispose", "error": { "code": "internal_error", @@ -6315,7 +8034,7 @@ }, { "kind": "error", - "id": 3607, + "id": 4107, "method": "session.dispose", "error": { "code": "cancelled", @@ -6325,7 +8044,7 @@ }, { "kind": "error", - "id": 3608, + "id": 4108, "method": "session.dispose", "error": { "code": "unknown_session", @@ -6335,7 +8054,7 @@ }, { "kind": "error", - "id": 3609, + "id": 4109, "method": "session.dispose", "error": { "code": "event_migration_required", @@ -6345,7 +8064,7 @@ }, { "kind": "error", - "id": 3610, + "id": 4110, "method": "session.dispose", "error": { "code": "invalid_idempotency_key", @@ -6355,7 +8074,7 @@ }, { "kind": "error", - "id": 3611, + "id": 4111, "method": "session.dispose", "error": { "code": "idempotency_conflict", @@ -6367,7 +8086,7 @@ "serverMessages": [ { "kind": "hello", - "wireVersion": 11, + "wireVersion": 12, "daemonInstanceId": "daemon-1", "capabilities": [ "session.create", diff --git a/packages/protocol/test/model-stream.test.ts b/packages/protocol/test/model-stream.test.ts new file mode 100644 index 0000000..70489ab --- /dev/null +++ b/packages/protocol/test/model-stream.test.ts @@ -0,0 +1,282 @@ +// SPDX-FileCopyrightText: 2026 Kaushik Kumar +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { + isTerminalModelStreamEvent, + ModelStreamValidationError, + parseModelStreamEvent, +} from "../src/index.ts"; + +const usage = { inputTokens: 3, outputTokens: 2, cacheReadTokens: 1, cacheWriteTokens: 0 }; + +test("validates positioned content and partial tool progress", () => { + assert.deepEqual(parseModelStreamEvent({ type: "text_delta", text: "answer", contentIndex: 2 }), { + type: "text_delta", + text: "answer", + contentIndex: 2, + }); + assert.deepEqual( + parseModelStreamEvent({ + type: "tool_call_start", + contentIndex: 3, + callId: "call-1", + name: "read", + }), + { type: "tool_call_start", contentIndex: 3, callId: "call-1", name: "read" }, + ); + assert.deepEqual( + parseModelStreamEvent({ + type: "tool_call_delta", + contentIndex: 3, + callId: "call-1", + argumentsDelta: '{"path":', + }), + { + type: "tool_call_delta", + contentIndex: 3, + callId: "call-1", + argumentsDelta: '{"path":', + }, + ); +}); + +test("validates provenance-bound replay metadata", () => { + const replay = parseModelStreamEvent({ + type: "replay_metadata", + target: "thinking", + contentIndex: 1, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + signature: '{"type":"reasoning","encrypted_content":"opaque"}', + responseId: "resp-1", + itemId: "rs-1", + }); + assert.deepEqual(replay, { + type: "replay_metadata", + target: "thinking", + contentIndex: 1, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + signature: '{"type":"reasoning","encrypted_content":"opaque"}', + responseId: "resp-1", + itemId: "rs-1", + }); + assert.equal(isTerminalModelStreamEvent(replay), false); + + assert.deepEqual( + parseModelStreamEvent({ + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "anthropic", + apiDialect: "anthropic-messages", + modelId: "claude-fixture", + signature: "opaque-redacted-thinking", + redacted: true, + }), + { + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "anthropic", + apiDialect: "anthropic-messages", + modelId: "claude-fixture", + signature: "opaque-redacted-thinking", + redacted: true, + }, + ); + + assert.deepEqual( + parseModelStreamEvent({ + type: "replay_metadata", + target: "tool_call", + contentIndex: 2, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + callId: "call-1", + itemId: "fc-1", + namespace: "tools", + }), + { + type: "replay_metadata", + target: "tool_call", + contentIndex: 2, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + callId: "call-1", + itemId: "fc-1", + namespace: "tools", + }, + ); +}); + +test("validates safe response attribution and retry guidance", () => { + const completed = parseModelStreamEvent({ + type: "completed", + stopReason: "stop", + usage, + partial: true, + response: { + providerId: "gateway", + requestedModelId: "auto", + routedModelId: "vendor/model", + responseId: "response-1", + nativeStopReason: "end_turn", + latencyMs: 12.5, + }, + diagnostics: [ + { code: "route_changed", message: "A routed model served the request", severity: "info" }, + ], + }); + assert.equal(isTerminalModelStreamEvent(completed), true); + + const failed = parseModelStreamEvent({ + type: "error", + code: "rate_limited", + message: "Try later", + retryable: true, + partial: false, + category: "rate_limit", + requestPhase: "awaiting_response", + retryAfterMs: 500, + retry: { retryAfterMs: 500, resetAtEpochMs: 2_000 }, + }); + assert.equal(isTerminalModelStreamEvent(failed), true); +}); + +test("rejects malformed stream data and unbounded diagnostic fields", () => { + assert.throws( + () => parseModelStreamEvent({ type: "text_delta", text: "x", contentIndex: -1 }), + ModelStreamValidationError, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "error", + code: "failed", + message: "failed", + retryable: false, + diagnostics: [ + { + code: "unsafe", + message: "unsafe", + severity: "error", + authorization: "Bearer secret", + }, + ], + }), + /authorization is not allowed/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "error", + code: "rate_limited", + message: "wait", + retryable: true, + retry: {}, + }), + /must contain retryAfterMs or resetAtEpochMs/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "error", + code: "failed", + message: "failed", + retryable: false, + category: "unsafe", + }), + /category is not recognized/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "error", + code: "failed", + message: "failed", + retryable: false, + requestPhase: "after_completion", + }), + /requestPhase is not recognized/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "tool_call", + callId: "call-1", + name: "read", + input: { invalid: Number.NaN }, + }), + /JSON-compatible/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "replay_metadata", + target: "thinking", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + }), + /must contain replay data/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "replay_metadata", + target: "text", + contentIndex: 0, + providerId: "anthropic", + apiDialect: "anthropic-messages", + modelId: "claude-fixture", + signature: "opaque", + redacted: true, + }), + /redacted is allowed only/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "replay_metadata", + target: "tool_call", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + itemId: "fc-1", + }), + /callId is required/, + ); + assert.throws( + () => + parseModelStreamEvent({ + type: "replay_metadata", + target: "text", + contentIndex: 0, + providerId: "openai", + apiDialect: "openai-responses", + modelId: "gpt-5", + callId: "call-1", + itemId: "msg-1", + }), + /callId is allowed only/, + ); +}); + +test("preserves the existing terminal event forms", () => { + assert.deepEqual(parseModelStreamEvent({ type: "completed", stopReason: "stop", usage }), { + type: "completed", + stopReason: "stop", + usage, + }); + assert.deepEqual(parseModelStreamEvent({ type: "aborted" }), { type: "aborted" }); +}); diff --git a/packages/protocol/test/version.test.ts b/packages/protocol/test/version.test.ts index 1d718b3..3f24852 100644 --- a/packages/protocol/test/version.test.ts +++ b/packages/protocol/test/version.test.ts @@ -8,7 +8,12 @@ import test from "node:test"; import { EVENT_FORMAT_VERSION, WIRE_PROTOCOL_VERSION } from "../src/index.ts"; -test("keeps event format 1 and adds model request configuration in wire protocol 11", () => { +test("keeps event format 1 and combines provider management with request settings in wire protocol 12", () => { assert.equal(EVENT_FORMAT_VERSION, 1); - assert.equal(WIRE_PROTOCOL_VERSION, 11); + assert.equal(WIRE_PROTOCOL_VERSION, 12); + assert.notEqual( + WIRE_PROTOCOL_VERSION, + 11, + "collided version 11 must be rejected by exact handshakes", + ); }); diff --git a/packages/protocol/test/wire.test.ts b/packages/protocol/test/wire.test.ts index a32a3a0..bcc12a4 100644 --- a/packages/protocol/test/wire.test.ts +++ b/packages/protocol/test/wire.test.ts @@ -11,6 +11,7 @@ import { encodeWireMessage, isRetryableMutationMethod, ProtocolValidationError, + parseProviderListResult, parseServerMessage, parseSnapshotPage, parseWireRequest, @@ -24,12 +25,58 @@ import { const sessionId = "123e4567-e89b-42d3-a456-426614174000"; +test("rejects secret-bearing provider inventory fields", () => { + assert.throws( + () => + parseProviderListResult({ + providers: [ + { + providerId: "openrouter", + displayName: "OpenRouter", + enabled: true, + authMethods: ["environment", "oauth"], + loginMethods: ["api_key", "oauth"], + authentication: { providerId: "openrouter", phase: "idle" }, + catalog: { refreshable: true }, + models: [], + apiKey: "must-not-cross-rpc", + }, + ], + }), + ProtocolValidationError, + ); +}); + +test("rejects secret-bearing provider error details", () => { + assert.throws( + () => + parseServerMessage({ + kind: "error", + id: 1, + method: "provider.auth.login", + error: { + code: "authentication_failed", + message: "Authentication failed", + retryable: false, + details: { + category: "authentication", + action: "login", + providerId: "openrouter", + token: "must-not-cross-rpc", + }, + }, + }), + ProtocolValidationError, + ); +}); + test("maps feature methods to negotiated capabilities", () => { assert.equal(requiredCapability("daemon.info"), undefined); assert.equal(requiredCapability("request.cancel"), undefined); assert.equal(requiredCapability("session.history"), undefined); assert.equal(requiredCapability("session.send"), "session.send.prompt"); assert.equal(requiredCapability("session.blob.abort"), "session.blob.abort"); + assert.equal(requiredCapability("provider.catalog.refresh"), "provider.catalog.refresh"); }); test("validates every request shape", () => { @@ -46,6 +93,31 @@ test("validates every request shape", () => { }, { kind: "request", id: 23, method: "connection.ping", params: {} }, { kind: "request", id: 24, method: "request.cancel", params: { requestId: 7 } }, + { kind: "request", id: 25, method: "provider.list", params: {} }, + { + kind: "request", + id: 26, + method: "provider.catalog.refresh", + params: { providerId: "openrouter" }, + }, + { + kind: "request", + id: 27, + method: "provider.auth.status", + params: { providerId: "openrouter" }, + }, + { + kind: "request", + id: 28, + method: "provider.auth.login", + params: { providerId: "openrouter", method: "oauth" }, + }, + { + kind: "request", + id: 29, + method: "provider.auth.logout", + params: { providerId: "openrouter" }, + }, { kind: "request", id: 1, diff --git a/packages/runtime/src/index.ts b/packages/runtime/src/index.ts index afb1de0..fd2698a 100644 --- a/packages/runtime/src/index.ts +++ b/packages/runtime/src/index.ts @@ -3,3 +3,4 @@ // SPDX-License-Identifier: Apache-2.0 export * from "./local-runtime.ts"; +export * from "./provider-management.ts"; diff --git a/packages/runtime/src/local-runtime.ts b/packages/runtime/src/local-runtime.ts index d3bc8aa..1e4862e 100644 --- a/packages/runtime/src/local-runtime.ts +++ b/packages/runtime/src/local-runtime.ts @@ -11,13 +11,20 @@ import type { CredentialStore } from "@axl/ai"; import { type AxlDaemon, listStoredSessions } from "@axl/daemon"; import { DEFAULT_MODEL_REQUEST_SETTINGS, - parseModelRequestSettings, type ModelRequestSettings, + parseModelRequestSettings, type SessionSummary, type ThinkingLevel, } from "@axl/protocol"; +import { + createProviderManagementService, + type TrustedProviderLoginAdapter, + validateProviderSelection, +} from "./provider-management.ts"; + export interface LocalRuntimeDefaults { + readonly providerId?: string; readonly requestSettings?: ModelRequestSettings; readonly modelId: string; readonly thinkingLevel: ThinkingLevel; @@ -148,6 +155,18 @@ export async function diagnoseLocalSandboxes(): Promise<{ }; } +async function migrateLegacyAzureCredential(store: CredentialStore): Promise { + const legacyProviderId = "azure-openai"; + const providerId = "azure-openai-responses"; + const [legacy, current] = await Promise.all([ + store.read(legacyProviderId), + store.read(providerId), + ]); + if (legacy === undefined || current !== undefined) return; + await store.modify(providerId, (stored) => Promise.resolve(stored ?? legacy)); + await store.delete(legacyProviderId); +} + async function exists(path: string): Promise { try { await access(path); @@ -158,6 +177,49 @@ async function exists(path: string): Promise { } } +export async function loginProviderFromTrustedHost(input: { + readonly store: CredentialStore; + readonly adapter: TrustedProviderLoginAdapter; + readonly providerId: string; + readonly axlHome: string; + readonly method: "api_key" | "oauth"; + readonly signal?: AbortSignal; +}): Promise { + const ai = await import("@axl/ai"); + const options = { store: input.store, context: ai.nodeAuthContext }; + const configured = await ai.loadConfiguredProviders(input.axlHome, options); + const providers = [ + ...ai + .createBuiltinProviders(options) + .filter((provider) => !configured.some((entry) => entry.id === provider.id)), + ...configured, + ]; + try { + const provider = providers.find((candidate) => candidate.id === input.providerId); + if (provider?.authentication === undefined) { + throw new Error(`Provider ${input.providerId} has no interactive authentication`); + } + if (!provider.authentication.loginMethods.includes(input.method)) { + throw new Error(`Provider ${input.providerId} does not support ${input.method} login`); + } + const signal = input.signal ?? new AbortController().signal; + const interaction = input.adapter.createInteraction({ + providerId: input.providerId, + method: input.method, + signal, + }); + const state = await provider.authentication.login(input.method, { ...interaction, signal }); + return { + providerId: input.providerId, + phase: state.phase, + ...(state.method === undefined ? {} : { method: state.method }), + ...(state.source === undefined ? {} : { source: state.source }), + }; + } finally { + await Promise.all(providers.map((provider) => provider.dispose?.())); + } +} + export interface LocalDaemonOptions { readonly buildVersion?: string; readonly onStopped?: () => void; @@ -183,7 +245,7 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise; + providers: import("@axl/ai").ProviderRegistry; }> | undefined; const loadAssembly = () => { @@ -203,12 +265,20 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise entry.id === provider.id)) providers.register(provider); + } + for (const provider of configured) providers.register(provider); + const restored = await providers.restoreCatalogs(); + return { ai, kernel, sandbox, providers, catalogErrors: restored.errors }; }); return assemblyPromise; }; @@ -217,6 +287,27 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise) => + createProviderManagementService((await loadAssembly()).providers).list(...args), + refresh: async ( + ...args: Parameters + ) => createProviderManagementService((await loadAssembly()).providers).refresh(...args), + authenticationStatus: async ( + ...args: Parameters + ) => + createProviderManagementService((await loadAssembly()).providers).authenticationStatus( + ...args, + ), + login: async (...args: Parameters) => + createProviderManagementService((await loadAssembly()).providers).login(...args), + logout: async ( + ...args: Parameters + ) => createProviderManagementService((await loadAssembly()).providers).logout(...args), + dispose: async () => { + if (assemblyPromise !== undefined) await (await assemblyPromise).providers.dispose(); + }, + } satisfies import("@axl/daemon").ProviderManagementService; const daemon = new AxlDaemon({ ...(options.buildVersion === undefined ? {} : { buildVersion: options.buildVersion }), ...(options.onStopped === undefined ? {} : { onStopped: options.onStopped }), @@ -226,8 +317,9 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise { - const { ai, kernel, sandbox, provider } = await loadAssembly(); + const { ai, kernel, sandbox, providers } = await loadAssembly(); const profile = selection.profile ?? "standard"; const [hasMcpConfig, hasSkills] = profile !== "standard" @@ -246,13 +338,7 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise candidate.modelId === active.modelId, + const modelInfo = await validateProviderSelection( + providers, + active.providerId, + active.modelId, ); - if (modelInfo === undefined) throw new Error(`Unknown Azure OpenAI model ${active.modelId}`); const thinking = ai.clampThinkingLevel(modelInfo, active.thinkingLevel); const policy = { workspace: cwd, @@ -280,11 +368,16 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise(); + const model = ai.modelPortForRegistry(providers, { + providerId: active.providerId, requestSettings, modelId: active.modelId, thinkingLevel: thinking.effective, readBlob, + onResolvedSecrets: (values) => { + for (const value of values) providerSecrets.add(value); + }, }); const tools = new kernel.ToolRegistry(); const overflowDirectory = join(stateDirectory, "tool-output"); @@ -351,13 +444,14 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise [ ...mcpSecrets, ...(braveSearchKey === undefined ? [] : [braveSearchKey]), + ...providerSecrets, ], }, sandbox: sandbox.configuredPayload(), + configProvider: { providerId: active.providerId }, configModel: { modelId: active.modelId }, configRequest: requestSettings, configThinking: thinking, @@ -367,7 +461,7 @@ export async function startLocalDaemon(options: LocalDaemonOptions): Promise { + if (level === "off") return model.thinkingLevelMap?.off !== null; + if ((level === "xhigh" || level === "max") && model.thinkingLevelMap?.[level] === undefined) { + return false; + } + return model.thinkingLevelMap?.[level] !== null; + }); +} + +function textModel(model: ModelInfo): ProviderTextModel { + return { + providerId: model.providerId, + modelId: model.modelId, + displayName: model.displayName, + apiDialect: model.apiDialect, + capabilities: { ...model.capabilities }, + reasoning: model.reasoning, + supportedThinkingLevels: supportedThinkingLevels(model), + contextWindow: model.contextWindow, + maxOutputTokens: model.maxOutputTokens, + ...(model.cost === undefined + ? {} + : { + cost: { + inputUsdPerMTok: model.cost.inputUsdPerMTok, + outputUsdPerMTok: model.cost.outputUsdPerMTok, + ...(model.cost.cacheReadUsdPerMTok === undefined + ? {} + : { cacheReadUsdPerMTok: model.cost.cacheReadUsdPerMTok }), + ...(model.cost.cacheWriteUsdPerMTok === undefined + ? {} + : { cacheWriteUsdPerMTok: model.cost.cacheWriteUsdPerMTok }), + }, + }), + availability: model.availability ?? { status: "available" }, + }; +} + +function errorDetails( + category: ProviderErrorCategory, + action: ProviderErrorAction, + providerId?: string, + modelId?: string, +) { + return { + category, + action, + ...(providerId === undefined ? {} : { providerId }), + ...(modelId === undefined ? {} : { modelId }), + }; +} + +function providerFailure( + code: ProviderRpcErrorCode, + message: string, + category: ProviderErrorCategory, + action: ProviderErrorAction, + providerId?: string, + modelId?: string, +): ProviderManagementError { + return new ProviderManagementError( + code, + message, + errorDetails(category, action, providerId, modelId), + ); +} + +function mapRegistryError(error: ProviderRegistryError): ProviderManagementError { + switch (error.code) { + case "provider_missing": + return providerFailure( + "provider_not_found", + "The requested provider is not registered", + "provider", + "configure_provider", + error.providerId, + ); + case "provider_disabled": + return providerFailure( + "provider_disabled", + "The requested provider is disabled", + "provider", + "configure_provider", + error.providerId, + ); + case "model_missing": + return providerFailure( + "model_not_found", + "The requested model is not present in the provider catalog", + "model", + "select_model", + error.providerId, + error.modelId, + ); + case "model_unavailable": + return providerFailure( + "model_unavailable", + "The requested model is currently unavailable", + "model", + "select_model", + error.providerId, + error.modelId, + ); + case "catalog_failure": + case "model_duplicate": + return providerFailure( + "catalog_refresh_failed", + "The provider catalog could not be validated", + "catalog", + "refresh_catalog", + error.providerId, + error.modelId, + ); + case "provider_duplicate": + case "registry_disposed": + return providerFailure( + "provider_configuration_required", + "Provider management is not available", + "configuration", + "configure_provider", + error.providerId, + ); + } +} + +function mapAuthError(error: AuthError): ProviderManagementError { + const lower = error.message.toLowerCase(); + if (lower.includes("region") || lower.includes("location")) { + const unsupported = lower.includes("invalid") || lower.includes("unsupported"); + return providerFailure( + unsupported ? "region_unsupported" : "region_required", + `Provider ${error.providerId} requires a valid region or location`, + "region", + "select_region", + error.providerId, + ); + } + if ( + lower.includes("project") || + lower.includes("account") || + lower.includes("gateway") || + lower.includes("base url") || + lower.includes("resource") + ) { + return providerFailure( + "provider_configuration_required", + `Provider ${error.providerId} requires additional configuration`, + "configuration", + "configure_provider", + error.providerId, + ); + } + if (error.code === "not_configured") { + return providerFailure( + "authentication_required", + `Provider ${error.providerId} requires authentication`, + "authentication", + "login", + error.providerId, + ); + } + return providerFailure( + "authentication_failed", + error.code === "refresh_failed" + ? `Provider ${error.providerId} requires login again` + : `Provider ${error.providerId} authentication failed`, + "authentication", + error.code === "refresh_failed" ? "logout_then_login" : "login", + error.providerId, + ); +} + +export function mapProviderManagementError(error: unknown): ProviderManagementError { + if (error instanceof ProviderManagementError) return error; + if (error instanceof ProviderRegistryError) return mapRegistryError(error); + if (error instanceof AuthError) return mapAuthError(error); + if (error instanceof DOMException && error.name === "AbortError") { + return providerFailure( + "authentication_failed", + "Provider operation was cancelled", + "provider", + "retry", + ); + } + return providerFailure( + "provider_configuration_required", + "Provider operation failed", + "configuration", + "configure_provider", + ); +} + +function refreshFailure( + providerId: string, + error: Error, +): ProviderCatalogRefreshResult["providers"][number] { + const mapped = mapProviderManagementError(error); + if (providerId === "github-copilot" && /(?:403|exhausted|quota)/i.test(error.message)) { + return { + providerId, + status: "failed", + modelCount: 0, + error: { + code: "entitlement_exhausted", + message: "GitHub Copilot access is unavailable for the current entitlement", + action: "configure_provider", + }, + }; + } + if (mapped.code === "authentication_required") { + return { + providerId, + status: "failed", + modelCount: 0, + error: { code: "authentication_required", message: mapped.message, action: "login" }, + }; + } + if (providerId === "github-copilot") { + return { + providerId, + status: "failed", + modelCount: 0, + error: { + code: "entitlement_required", + message: "GitHub Copilot catalog access requires an active entitlement", + action: "login", + }, + }; + } + return { + providerId, + status: "failed", + modelCount: 0, + error: { + code: "catalog_refresh_failed", + message: "The provider catalog could not be refreshed", + action: "retry", + }, + }; +} + +/** Adapts the AI registry to the daemon's credential-free provider contract. */ +export function createProviderManagementService( + registry: ProviderRegistry, + options: { readonly loginAdapter?: TrustedProviderLoginAdapter } = {}, +): ProviderManagementService { + const registration = (providerId: string) => { + const found = registry.registrations().find((entry) => entry.provider.id === providerId); + if (found === undefined) { + throw providerFailure( + "provider_not_found", + "The requested provider is not registered", + "provider", + "configure_provider", + providerId, + ); + } + if (!found.enabled) { + throw providerFailure( + "provider_disabled", + "The requested provider is disabled", + "provider", + "configure_provider", + providerId, + ); + } + return found; + }; + + return { + list: async (params, signal) => { + signal?.throwIfAborted(); + const metadata = new Map(listBuiltinCatalogProviders().map((item) => [item.id, item])); + const registrations = + params.providerId === undefined + ? registry.registrations() + : [registration(params.providerId)]; + const providers: ProviderInventoryGroup[] = []; + for (const entry of registrations) { + signal?.throwIfAborted(); + const provider = entry.provider; + const snapshot = registry.catalogSnapshot(provider.id); + let models: readonly ModelInfo[] = []; + let catalogError: ProviderInventoryGroup["catalogError"]; + if (entry.enabled) { + const listed = await registry.listModels({ + providerId: provider.id, + includeUnavailable: true, + }); + models = listed.models; + if (listed.errors.has(provider.id)) { + catalogError = { + code: "catalog_failure", + message: "The provider catalog could not be listed", + action: + provider.refreshModelCatalog === undefined && provider.refreshModels === undefined + ? "configure_provider" + : "refresh_catalog", + }; + } + } + const catalogMetadata = metadata.get(provider.id); + providers.push({ + providerId: provider.id, + displayName: provider.displayName, + enabled: entry.enabled, + ...(catalogMetadata?.regionFamily === undefined + ? {} + : { regionFamily: catalogMetadata.regionFamily }), + ...(catalogMetadata?.region === undefined ? {} : { region: catalogMetadata.region }), + authMethods: [...provider.authMethods], + loginMethods: loginMethods(provider), + authentication: authenticationStatus(provider), + catalog: { + refreshable: + provider.refreshModelCatalog !== undefined || provider.refreshModels !== undefined, + ...(snapshot === undefined + ? {} + : { + generation: snapshot.generation, + checkedAt: snapshot.checkedAt, + updatedAt: snapshot.updatedAt, + source: { ...snapshot.source }, + }), + }, + models: models.map(textModel), + ...(catalogError === undefined ? {} : { catalogError }), + }); + } + return { providers }; + }, + refresh: async (params, signal) => { + signal?.throwIfAborted(); + if (params.providerId !== undefined) { + const { provider } = registration(params.providerId); + if (provider.refreshModelCatalog === undefined && provider.refreshModels === undefined) { + throw providerFailure( + "catalog_refresh_unsupported", + `Provider ${provider.id} has a static catalog`, + "catalog", + "configure_provider", + provider.id, + ); + } + } + try { + const result = await registry.refresh({ + configuredOnly: params.providerId === undefined, + ...(params.providerId === undefined ? {} : { providerId: params.providerId }), + ...(signal === undefined ? {} : { signal }), + }); + signal?.throwIfAborted(); + const selected = + params.providerId === undefined + ? registry + .registrations() + .filter( + (entry) => + entry.enabled && + (entry.provider.refreshModelCatalog !== undefined || + entry.provider.refreshModels !== undefined), + ) + : [registration(params.providerId)]; + const providers = selected + .filter( + ({ provider }) => + result.errors.has(provider.id) || + result.refreshedProviderIds.includes(provider.id) || + result.supersededProviderIds.includes(provider.id), + ) + .map(({ provider }) => { + const error = result.errors.get(provider.id); + if (error !== undefined) return refreshFailure(provider.id, error); + const snapshot = + result.snapshots.get(provider.id) ?? registry.catalogSnapshot(provider.id); + return { + providerId: provider.id, + status: result.supersededProviderIds.includes(provider.id) + ? ("superseded" as const) + : result.refreshedProviderIds.includes(provider.id) + ? ("refreshed" as const) + : ("not_modified" as const), + modelCount: snapshot?.models.length ?? 0, + }; + }); + if (params.providerId !== undefined) { + const failure = providers[0]; + if (failure?.status === "failed" && failure.error !== undefined) { + const failureError = failure.error; + throw providerFailure( + failureError.code, + failureError.message, + failureError.code === "entitlement_required" || + failureError.code === "entitlement_exhausted" + ? "entitlement" + : "catalog", + failureError.action, + failure.providerId, + ); + } + } + return { providers }; + } catch (error) { + if (signal?.aborted) signal.throwIfAborted(); + throw mapProviderManagementError(error); + } + }, + authenticationStatus: async (params, signal) => { + signal?.throwIfAborted(); + const registrations = + params.providerId === undefined + ? registry.registrations().filter((entry) => entry.enabled) + : [registration(params.providerId)]; + const providers: ProviderAuthenticationStatus[] = []; + for (const { provider } of registrations) { + signal?.throwIfAborted(); + try { + const state = await provider.authentication?.check( + signal === undefined ? {} : { signal }, + ); + providers.push(authenticationStatus(provider, state)); + } catch (error) { + if (signal?.aborted) signal.throwIfAborted(); + if (params.providerId !== undefined) throw mapProviderManagementError(error); + providers.push({ + providerId: provider.id, + phase: "reauthentication_required", + }); + } + } + return { providers }; + }, + login: async (params, signal) => { + const { provider } = registration(params.providerId); + const authentication = provider.authentication; + if (authentication === undefined || !loginMethods(provider).includes(params.method)) { + throw providerFailure( + "authentication_unavailable", + `Provider ${provider.id} does not support ${params.method} login`, + "authentication", + "configure_provider", + provider.id, + ); + } + if (options.loginAdapter === undefined) { + throw providerFailure( + "authentication_unavailable", + "Interactive login requires a trusted process-host adapter", + "authentication", + "configure_provider", + provider.id, + ); + } + const effectiveSignal = signal ?? new AbortController().signal; + try { + const interaction = options.loginAdapter.createInteraction({ + providerId: provider.id, + method: params.method, + signal: effectiveSignal, + }); + const state = await authentication.login(params.method, { + ...interaction, + signal: effectiveSignal, + }); + return authenticationStatus(provider, state); + } catch (error) { + if (effectiveSignal.aborted) effectiveSignal.throwIfAborted(); + throw mapProviderManagementError(error); + } + }, + logout: async (params, signal) => { + const { provider } = registration(params.providerId); + if (provider.authentication === undefined) { + throw providerFailure( + "authentication_unavailable", + `Provider ${provider.id} has no stored authentication lifecycle`, + "authentication", + "configure_provider", + provider.id, + ); + } + try { + return authenticationStatus( + provider, + await provider.authentication.logout({ ...(signal === undefined ? {} : { signal }) }), + ); + } catch (error) { + if (signal?.aborted) signal.throwIfAborted(); + throw mapProviderManagementError(error); + } + }, + dispose: () => registry.dispose(), + }; +} + +/** Validates a selection and configured authentication before it becomes session state. */ +export async function validateProviderSelection( + registry: ProviderRegistry, + providerId: string, + modelId: string, + signal?: AbortSignal, +): Promise { + try { + const model = await registry.getModel(providerId, modelId); + const authentication = registry.get(providerId).authentication; + if (authentication !== undefined) { + const status = await authentication.check({ ...(signal === undefined ? {} : { signal }) }); + if (status.phase !== "authenticated") { + throw providerFailure( + "authentication_required", + `Provider ${providerId} requires authentication before selecting ${modelId}`, + "authentication", + "login", + providerId, + modelId, + ); + } + } + return model; + } catch (error) { + if (signal?.aborted) signal.throwIfAborted(); + throw mapProviderManagementError(error); + } +} diff --git a/packages/runtime/test/local-runtime.test.ts b/packages/runtime/test/local-runtime.test.ts index 91e9719..f1e346c 100644 --- a/packages/runtime/test/local-runtime.test.ts +++ b/packages/runtime/test/local-runtime.test.ts @@ -5,18 +5,149 @@ // SPDX-License-Identifier: Apache-2.0 import assert from "node:assert/strict"; -import { mkdir, mkdtemp, rm } from "node:fs/promises"; +import { mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; +import { createServer } from "node:http"; import { tmpdir } from "node:os"; import { join } from "node:path"; import test from "node:test"; -import { FileCredentialStore } from "@axl/ai"; +import { FileCredentialStore, getStaticModelCatalog } from "@axl/ai"; import { AxlDaemon } from "@axl/daemon"; import { type ModelPort, ToolRegistry } from "@axl/kernel"; import type { ModelStreamEvent } from "@axl/protocol"; +import { AxlClientError } from "@axl/sdk"; import { connectUnixClient } from "@axl/sdk/unix"; -import { listLocalSessions, localSandboxStateKey, startLocalDaemon } from "../src/index.ts"; +import { + listLocalSessions, + localSandboxStateKey, + loginProviderFromTrustedHost, + startLocalDaemon, +} from "../src/index.ts"; + +test("provider output cannot persist rotating request credentials", async (context) => { + const root = await mkdtemp(join(tmpdir(), "axl-runtime-redaction-")); + context.after(() => rm(root, { recursive: true, force: true })); + const axlHome = join(root, ".axl"); + const workspace = join(root, "workspace"); + const stateDirectory = join(axlHome, "unsafe"); + await mkdir(workspace, { recursive: true }); + let secret = "first-provider-secret"; + let requests = 0; + const server = createServer((_request, response) => { + requests += 1; + response.writeHead(200, { "content-type": "text/event-stream" }); + const events = + requests % 2 === 1 + ? [ + { + type: "response.output_item.added", + output_index: 0, + item: { type: "message", id: `m-${requests}` }, + }, + { type: "response.output_text.delta", output_index: 0, delta: secret }, + { + type: "response.output_item.done", + output_index: 0, + item: { type: "message", id: `m-${requests}`, content: [] }, + }, + { + type: "response.output_item.added", + output_index: 1, + item: { + type: "function_call", + id: `f-${requests}`, + call_id: `call-${requests}`, + name: "read", + }, + }, + { + type: "response.function_call_arguments.delta", + output_index: 1, + delta: JSON.stringify({ path: secret }), + }, + { + type: "response.output_item.done", + output_index: 1, + item: { + type: "function_call", + id: `f-${requests}`, + call_id: `call-${requests}`, + name: "read", + arguments: JSON.stringify({ path: secret }), + }, + }, + { + type: "response.completed", + response: { id: `r-${requests}`, status: "completed", usage: {} }, + }, + ] + : [ + { + type: "response.completed", + response: { id: `r-${requests}`, status: "completed", usage: {} }, + }, + ]; + response.end( + `${events.map((event) => `data: ${JSON.stringify(event)}\n\n`).join("")}data: [DONE]\n\n`, + ); + }); + await new Promise((resolvePromise) => server.listen(0, "127.0.0.1", resolvePromise)); + context.after(() => new Promise((resolvePromise) => server.close(() => resolvePromise()))); + const address = server.address(); + if (address === null || typeof address === "string") throw new Error("test server has no port"); + const source = getStaticModelCatalog("openai").find( + (model) => model.apiDialect === "openai-responses", + ); + if (source === undefined) throw new Error("OpenAI Responses catalog is empty"); + await mkdir(axlHome, { recursive: true }); + await writeFile( + join(axlHome, "models.json"), + JSON.stringify({ + providers: { + custom: { + baseUrl: `http://127.0.0.1:${address.port}/v1`, + apiKeyEnvironmentVariables: ["AXL_TEST_CUSTOM_KEY"], + models: [{ ...source, providerId: "custom", modelId: "echo-model" }], + }, + }, + }), + ); + process.env.AXL_TEST_CUSTOM_KEY = secret; + context.after(() => delete process.env.AXL_TEST_CUSTOM_KEY); + const socketPath = join(stateDirectory, "axl.sock"); + const daemon = await startLocalDaemon({ + axlHome, + stateDirectory, + socketPath, + defaults: { providerId: "custom", modelId: "echo-model", thinkingLevel: "off" }, + store: new FileCredentialStore(join(axlHome, "credentials.json")), + unsafe: true, + }); + context.after(() => daemon.stop()); + const client = await connectUnixClient(socketPath); + context.after(() => client.close()); + const created = await client.request("session.create", { cwd: workspace }); + await client.request("session.send", { + sessionId: created.sessionId, + delivery: "prompt", + content: [{ type: "text", text: "first" }], + }); + secret = "rotated-provider-secret"; + process.env.AXL_TEST_CUSTOM_KEY = secret; + await client.request("session.send", { + sessionId: created.sessionId, + delivery: "prompt", + content: [{ type: "text", text: "second" }], + }); + const raw = await readFile( + join(stateDirectory, "sessions", `${created.sessionId}.jsonl`), + "utf8", + ); + assert.equal(raw.includes("first-provider-secret"), false); + assert.equal(raw.includes("rotated-provider-secret"), false); + assert.equal(raw.includes("[REDACTED]"), true); +}); test("OCI state keys require a digest and cannot traverse directories", () => { assert.equal( @@ -92,6 +223,20 @@ test("assembles an authoritative local runtime without a presentation client", a await mkdir(workspace, { recursive: true }); const store = new FileCredentialStore(join(axlHome, "credentials.json")); + const customSource = getStaticModelCatalog("deepseek")[0]; + if (customSource === undefined) throw new Error("DeepSeek catalog is empty"); + await mkdir(axlHome, { recursive: true }); + await writeFile( + join(axlHome, "models.json"), + JSON.stringify({ + providers: { + custom: { + baseUrl: "http://127.0.0.1:11434/v1", + models: [{ ...customSource, providerId: "custom", modelId: "local-model" }], + }, + }, + }), + ); await store.modify("azure-openai", () => Promise.resolve({ type: "api_key", @@ -116,6 +261,83 @@ test("assembles an authoritative local runtime without a presentation client", a securityMode: "unsafe", sandboxProvider: "none", }); + const allProviders = await client.listProviders(); + assert.equal(allProviders.providers.length, 41); + assert.deepEqual( + allProviders.providers + .find((provider) => provider.providerId === "custom") + ?.models.map((model) => model.modelId), + ["local-model"], + ); + const inventory = await client.listProviders({ providerId: "azure-openai-responses" }); + assert.equal(inventory.providers.length, 1); + assert.deepEqual(inventory.providers[0]?.loginMethods, ["api_key"]); + assert.equal( + inventory.providers[0]?.models.some((model) => model.modelId === "gpt-5"), + true, + ); + assert.equal(JSON.stringify(inventory).includes("obviously-fake-runtime-test-key"), false); + assert.deepEqual( + await client.providerAuthenticationStatus({ providerId: "azure-openai-responses" }), + { + providers: [ + { + providerId: "azure-openai-responses", + phase: "authenticated", + method: "api_key", + source: "Azure OpenAI API key", + }, + ], + }, + ); + await assert.rejects( + client.request("session.create", { + cwd: workspace, + providerId: "missing-provider", + modelId: "missing-model", + }), + (error) => + error instanceof AxlClientError && + error.code === "provider_not_found" && + error.details?.action === "configure_provider", + ); + await assert.rejects( + client.loginProvider({ providerId: "deepseek", method: "api_key" }), + (error) => error instanceof AxlClientError && error.code === "authentication_unavailable", + ); + const prompts = { requesting: 0, other: 0 }; + const login = await loginProviderFromTrustedHost({ + store, + axlHome, + providerId: "deepseek", + method: "api_key", + adapter: { + createInteraction: () => ({ + prompt: async () => { + prompts.requesting += 1; + return "runtime-login-secret"; + }, + notify: () => {}, + }), + }, + }); + const unrelatedAdapter = { + createInteraction: () => ({ + prompt: async () => { + prompts.other += 1; + return "wrong-client-secret"; + }, + notify: () => {}, + }), + }; + void unrelatedAdapter; + assert.equal(login.phase, "authenticated"); + assert.deepEqual(prompts, { requesting: 1, other: 0 }); + assert.equal(JSON.stringify(login).includes("runtime-login-secret"), false); + assert.deepEqual(await client.logoutProvider({ providerId: "deepseek" }), { + providerId: "deepseek", + phase: "logged_out", + }); const opened = await client.request("session.create", { cwd: workspace }); const subscription = await client.request("session.subscribe", { sessionId: opened.sessionId, diff --git a/packages/sdk/src/client.ts b/packages/sdk/src/client.ts index 23b6920..aaaa289 100644 --- a/packages/sdk/src/client.ts +++ b/packages/sdk/src/client.ts @@ -5,6 +5,7 @@ import { encodeWireMessage, isKnownRpcErrorCode, + isProviderRpcErrorCode, isRetryableMutationMethod, isRpcErrorAllowed, parseServerMessage, @@ -16,6 +17,8 @@ import { type ConnectionInitializeResult, type OperationId, type PresenceDelivery, + type ProviderRpcErrorCode, + type ProviderRpcErrorDetails, type RpcMethod, type RpcParams, type RpcResult, @@ -86,6 +89,25 @@ export class AxlClientError extends Error { } } +export class ProviderClientError extends AxlClientError { + declare readonly code: ProviderRpcErrorCode; + declare readonly details: ProviderRpcErrorDetails; + + constructor( + code: ProviderRpcErrorCode, + message: string, + options: { + readonly retryable: boolean; + readonly details: ProviderRpcErrorDetails; + }, + ) { + super(code, message, options); + this.name = "ProviderClientError"; + this.code = code; + this.details = options.details; + } +} + export interface AxlClientOptions { readonly transport: AxlTransportFactory; readonly identity: ClientIdentity; @@ -229,6 +251,41 @@ export class AxlClient { }); } + listProviders( + params: RpcParams<"provider.list"> = {}, + options: Omit = {}, + ): Promise> { + return this.request("provider.list", params, options); + } + + refreshProviderCatalogs( + params: RpcParams<"provider.catalog.refresh"> = {}, + options: Omit = {}, + ): Promise> { + return this.request("provider.catalog.refresh", params, options); + } + + providerAuthenticationStatus( + params: RpcParams<"provider.auth.status"> = {}, + options: Omit = {}, + ): Promise> { + return this.request("provider.auth.status", params, options); + } + + loginProvider( + params: RpcParams<"provider.auth.login">, + options: Omit = {}, + ): Promise> { + return this.request("provider.auth.login", params, options); + } + + logoutProvider( + params: RpcParams<"provider.auth.logout">, + options: Omit = {}, + ): Promise> { + return this.request("provider.auth.logout", params, options); + } + async shell( params: RpcParams<"session.shell">, options: Omit = {}, @@ -487,10 +544,15 @@ export class AxlClient { return; } } - const error = new AxlClientError(message.error.code, message.error.message, { - retryable: message.error.retryable, - ...(message.error.details === undefined ? {} : { details: message.error.details }), - }); + const error = isProviderRpcErrorCode(message.error.code) + ? new ProviderClientError(message.error.code, message.error.message, { + retryable: message.error.retryable, + details: message.error.details as ProviderRpcErrorDetails, + }) + : new AxlClientError(message.error.code, message.error.message, { + retryable: message.error.retryable, + ...(message.error.details === undefined ? {} : { details: message.error.details }), + }); if (message.id === -1) this.fail(error); else this.rejectRequest(message.id, error); } else if (message.kind === "event") { diff --git a/packages/sdk/src/models.ts b/packages/sdk/src/models.ts index 1c01643..350482b 100644 --- a/packages/sdk/src/models.ts +++ b/packages/sdk/src/models.ts @@ -12,6 +12,7 @@ export interface ClientModelCost { /** Provider-neutral model metadata used by presentation clients. */ export interface ClientModelInfo { + readonly providerId?: string; readonly modelId: string; readonly displayName: string; readonly reasoning: boolean; diff --git a/packages/sdk/test/client.test.ts b/packages/sdk/test/client.test.ts index 08aff9d..3512142 100644 --- a/packages/sdk/test/client.test.ts +++ b/packages/sdk/test/client.test.ts @@ -9,6 +9,7 @@ import { AxlClient, AxlClientError, type AxlTransport, + ProviderClientError, type AxlTransportFactory, } from "../src/index.ts"; import { @@ -146,6 +147,65 @@ test("initializes exactly once and creates keys only for retryable mutations", a client.close(); }); +test("exposes typed provider methods through negotiated capabilities", async () => { + const { client, transport } = await connect(); + const pending = client.listProviders({ providerId: "openrouter" }); + const request = transport.messages.at(-1) as { + id: number; + method: string; + params: Record; + }; + assert.equal(request.method, "provider.list"); + assert.deepEqual(request.params, { providerId: "openrouter" }); + transport.emit({ + kind: "success", + id: request.id, + method: "provider.list", + result: { providers: [] }, + }); + assert.deepEqual(await pending, { providers: [] }); + client.close(); +}); + +test("does not replay provider actions after reconnect", async () => { + const factory = new Factory(); + const { client, transport } = await connect(factory); + const pending = client.refreshProviderCatalogs({ providerId: "openrouter" }); + transport.closeListener?.(new Error("lost response")); + await assert.rejects( + pending, + (error) => error instanceof AxlClientError && error.code === "disconnected", + ); + assert.equal(factory.transports.length, 1); + client.close(); +}); + +test("returns provider failures as typed actionable SDK errors", async () => { + const { client, transport } = await connect(); + const pending = client.providerAuthenticationStatus({ providerId: "openrouter" }); + const request = transport.messages.at(-1) as { id: number }; + transport.emit({ + kind: "error", + id: request.id, + method: "provider.auth.status", + error: { + code: "authentication_failed", + message: "Provider authentication failed", + retryable: false, + details: { category: "authentication", action: "login", providerId: "openrouter" }, + }, + }); + await assert.rejects( + pending, + (error) => + error instanceof ProviderClientError && + error.code === "authentication_failed" && + error.details.providerId === "openrouter" && + error.details.action === "login", + ); + client.close(); +}); + test("preserves unknown future structured errors without crashing", async () => { const { client, transport } = await connect(); const pending = client.request("daemon.info", {}); diff --git a/packages/tui/README.md b/packages/tui/README.md index 08c52fb..7b0bbcf 100644 --- a/packages/tui/README.md +++ b/packages/tui/README.md @@ -89,6 +89,16 @@ Consecutive tool calls retain an individual shaded block for every call, with gr Run `/commands` for searchable actions, `/hotkeys` for keybindings, `/details` for compact, full, or focus transcript presentation, and `/settings` for persistent terminal preferences. `/prompt` browses reusable Markdown prompts from `~/.axl/prompts/` and `.axl/prompts/`; `/prompt [arguments]` expands one into the editor, and `/reload` reloads template files. `/compact [instructions]` shows cancellable progress, then a collapsed summary. Ctrl plus O expands the summary. Replaced messages leave the active transcript but remain in JSONL, `/export`, and any existing native scrollback. Repeating compaction reports `Already compacted`; insufficient context reports `Nothing to compact (session too small)`. `/export [directory]` writes a portable session artifact, and `/import ` validates that artifact and opens it as a new session in the current workspace. `/stash` preserves or swaps a draft, `/favorite` manages model favorites, `/developer` toggles the wide workspace panel, `/vim` toggles Vim editing, and `/review` opens bounded workspace review. Workspace checkpoints remain off until review is enabled and can be disabled with `/review off`. +## Model provider workflows + +`/model` opens a grouped, favorite-first text-model picker. Models are identified by the canonical provider and model pair. Use `/model /` for an exact selection. Unavailable models remain visible with their reason, and the daemon validates every accepted selection. API dialect appears only as explanatory model metadata. + +`/providers [provider-id]` shows explicit authentication and catalog status. `/login` first offers account or API-key sign-in, then a searchable provider list with configuration status. `/login ` selects that provider directly. Provider prompts, masked secrets, browser instructions, and device codes stay in an inline dialog. Answers pass directly to the trusted process host and never enter daemon RPC, canonical events, or transcript history. `/logout [provider-id]` removes stored authentication. `/refresh [provider-id]` explicitly refreshes configured provider catalogs, including models.dev-backed built-ins. Escape or Ctrl+C cancels an active login. + +The editor status reports usage for the last completed turn and cumulative session usage. It includes input, output, cache, and reasoning tokens plus USD cost when available. Provider-reported cost is used first. Catalog pricing supplies a provider-qualified presentation estimate only when the turn reports usage without cost. + +Provider failures remain visible with a safe message, category, provider and model identity when available, concrete action, and retry guidance. Login, logout, and refresh are never replayed automatically after reconnect. + Ctrl plus V pastes an image or text. Images are saved to owner-only `axl-clipboard-` files in the operating system's temporary directory, and their paths appear in the draft. On submission, intact standalone paths created by this client are uploaded through the daemon blob channel. Delete a path from the draft to omit that image. Temp files remain available for reuse until the operating system removes them; use `/attach ` to reuse one after restarting Axl. Clipboard reading uses `wl-paste` on Wayland, `xclip` on X11, AppKit through `osascript` on macOS, and PowerShell on Windows or WSL. Missing helpers and unsupported image formats produce visible errors. During a model turn, Enter sends steering after the current complete tool-call batch and Alt plus Enter queues a follow-up after the turn would otherwise finish. Pending messages from this terminal appear above the editor in numbered injection order: steering FIFO, then follow-up FIFO. Consumed messages disappear and the remaining positions update. This list does not include pending steering from other clients. Dropping image paths attaches them to the next prompt. `/attach ` provides an explicit keyboard flow, while `/attach clear` removes pending attachments and clears clipboard-path recognition. Image display can be set to auto, inline, or metadata in `/settings`. See `docs/terminal-compatibility.md` for capability overrides and the manual terminal matrix. diff --git a/packages/tui/src/app.ts b/packages/tui/src/app.ts index 2dbb6a5..546d7d7 100644 --- a/packages/tui/src/app.ts +++ b/packages/tui/src/app.ts @@ -23,6 +23,10 @@ import type { EventPayloadMap, JsonObject, JsonValue, + ProviderAuthenticationStatus, + ProviderInventoryGroup, + ProviderLoginMethod, + ProviderTextModel, SessionId, SessionOpenResult, SessionProfile, @@ -33,13 +37,14 @@ import { parseEventId, parseOperationId, parseSessionId } from "@axl/protocol"; import { type AxlClient, AxlClientError, + type ClientModelInfo, + ConversationProjector, type DaemonHostControl, type DaemonHostStatus, type ModelRequestSettings, - parseModelRequestSettings, - type ClientModelInfo, - ConversationProjector, orderPendingTurnInputs, + ProviderClientError, + parseModelRequestSettings, type SessionSubscription, subscribeSession, supportedThinkingLevels, @@ -62,7 +67,7 @@ import { decodeOneKey, LineEditor } from "./editor.ts"; import { EditorFrameComponent } from "./editor-frame.ts"; import { ExtensionWidgetsComponent } from "./extension-ui.ts"; import { editPromptExternally } from "./external-editor.ts"; -import { fullscreenDockHeight, type FullscreenMouse, FullscreenScreen } from "./fullscreen.ts"; +import { type FullscreenMouse, FullscreenScreen, fullscreenDockHeight } from "./fullscreen.ts"; import { isMouseReport } from "./fullscreen-input.ts"; import { LiveAssistantComponent } from "./live-assistant.ts"; import type { LoginDialogDefinition } from "./login-dialog.ts"; @@ -76,12 +81,13 @@ import { } from "./media.ts"; import { type Overlay, OverlayStack } from "./overlay.ts"; import { PickerOverlay } from "./picker.ts"; +import { ProviderLoginOverlay, type ProviderLoginPresentation } from "./provider-login.ts"; import { AUTOWRAP_OFF, AUTOWRAP_ON, - clipFrame, type Component, type CursorPlacement, + clipFrame, DifferentialScreen, SYNC_BEGIN, SYNC_END, @@ -164,6 +170,39 @@ function orderSessions(sessions: readonly T[]): T[] { return [...sessions].sort((left, right) => right.updatedAt - left.updatedAt); } +function providerErrorText(error: unknown): string { + if (!(error instanceof ProviderClientError)) { + return error instanceof Error + ? sanitizeTerminalText(error.message) + : "provider operation failed"; + } + const subject = [error.details.providerId, error.details.modelId].filter(Boolean).join("/"); + return [ + sanitizeTerminalText(error.message), + subject ? `${error.details.category}: ${subject}` : error.details.category, + `action: ${error.details.action.replaceAll("_", " ")}`, + ...(error.retryable ? ["retryable"] : []), + ].join(" · "); +} + +function authenticationLabel(status: ProviderAuthenticationStatus): string { + return [status.phase.replaceAll("_", " "), status.method, status.source] + .filter(Boolean) + .join(" · "); +} + +function formatProviderModel(model: ProviderTextModel): ClientModelInfo { + return { + providerId: model.providerId, + modelId: model.modelId, + displayName: model.displayName, + reasoning: model.reasoning, + contextWindow: model.contextWindow, + maxOutputTokens: model.maxOutputTokens, + ...(model.cost === undefined ? {} : { cost: model.cost }), + }; +} + function formatPath(cwd: string): string { const home = resolve(homedir()); const path = resolve(cwd); @@ -308,14 +347,17 @@ function openExternalUrl(url: string, onError: (error: Error) => void): void { } const COMMANDS: readonly { readonly name: string; readonly summary: string }[] = [ - { name: "/model", summary: "select a model, or /model " }, + { name: "/model", summary: "select a model grouped by provider" }, { name: "/thinking", summary: "select reasoning effort" }, { name: "/theme", summary: "select a color theme" }, { name: "/settings", summary: "change persistent terminal preferences" }, { name: "/details", summary: "set transcript detail: compact, full, or focus" }, { name: "/fullscreen", summary: "switch to fullscreen transcript mode" }, { name: "/regular", summary: "return to terminal scrollback mode" }, - { name: "/login", summary: "configure provider credentials" }, + { name: "/providers", summary: "show provider authentication and catalog status" }, + { name: "/login", summary: "authenticate a provider" }, + { name: "/logout", summary: "remove stored provider authentication" }, + { name: "/refresh", summary: "refresh configured provider catalogs" }, { name: "/reload", summary: "reload AGENTS.md, prompt, and tools" }, { name: "/compact", summary: "summarize older context, optionally with instructions" }, { name: "/status", summary: "show session, display, and queue state" }, @@ -457,6 +499,7 @@ export interface AxlAppOptions { readonly globalThemeDirectory?: string; readonly models?: readonly string[]; readonly modelCatalog?: readonly ClientModelInfo[]; + readonly currentProvider?: string; readonly currentModel?: string; readonly currentThinking?: ThinkingLevel; readonly profile?: SessionProfile; @@ -479,6 +522,7 @@ export interface AxlAppOptions { readonly mediaCapabilities?: TerminalMediaCapabilities; readonly extensions?: readonly TerminalExtension[]; readonly onPreferenceChange?: (update: { + providerId?: string; modelId?: string; thinkingLevel?: ThinkingLevel; requestSettings?: ModelRequestSettings; @@ -503,6 +547,14 @@ export interface AxlAppOptions { /** Compatibility hook called after the daemon accepts a model switch. */ readonly onModelChange?: (modelId: string) => void; readonly suspendProcess?: () => void; + /** Provider authentication performed by the invoking trusted process host. */ + readonly loginProvider?: ( + providerId: string, + method: ProviderLoginMethod, + signal: AbortSignal, + presentation: ProviderLoginPresentation, + ) => Promise; + /** Legacy process-host dialog retained for compatibility attachments. */ readonly loadLogin?: () => Promise; readonly onExit?: () => void; readonly clearStartupLine?: boolean; @@ -586,9 +638,12 @@ export class AxlApp { readonly text: string; }> = []; private sending = false; + private awaitingOperationOwnership = false; private interrupting = false; private activeRequest: "turn" | "shell" | "compaction" | undefined; private configuring = false; + private providerOperation: AbortController | undefined; + private providerInventory: readonly ProviderInventoryGroup[] = []; private webFetchEnabled: boolean; private webSearchEnabled: boolean; private initialResumePending: boolean; @@ -905,6 +960,9 @@ export class AxlApp { : options.sessionId === undefined ? await options.client.request("session.create", { cwd: options.cwd, + ...(options.currentProvider === undefined + ? {} + : { providerId: options.currentProvider }), ...(options.requestSettings === undefined ? {} : { requestSettings: options.requestSettings }), @@ -1025,6 +1083,8 @@ export class AxlApp { this.stopThemeWatcher = undefined; for (const controller of this.extensionCommandControllers) controller.abort(); this.extensionCommandControllers.clear(); + this.providerOperation?.abort(); + this.providerOperation = undefined; const failures: unknown[] = []; const extensionCleanup = this.extensionHost.dispose(); @@ -1190,7 +1250,7 @@ export class AxlApp { lines, this.tuiMode === "regular" ? this.height : fullscreenDockHeight(this.height), cursor === undefined ? undefined : { ...cursor, row: prefix.length + cursor.row }, - prefix.length, + prefix.length + (this.overlays.active instanceof PickerOverlay ? 4 : 0), ); } @@ -1330,32 +1390,42 @@ export class AxlApp { } } const argument = - /^(\/model|\/thinking|\/theme|\/details|\/favorite|\/developer|\/review|\/vim)\s+(\S*)$/.exec( + /^(\/model|\/providers|\/login|\/logout|\/refresh|\/thinking|\/theme|\/details|\/favorite|\/developer|\/review|\/vim)\s+(\S*)$/.exec( text, ); if (!argument) return []; const [, command, query = ""] = argument; const values = command === "/model" - ? (this.options.models ?? []) - : command === "/thinking" - ? (() => { - const model = this.options.modelCatalog?.find( - (candidate) => candidate.modelId === this.view.model, - ); - return model === undefined ? THINKING_LEVELS : supportedThinkingLevels(model); - })() - : command === "/theme" - ? themeNames(this.themeDefinitions) - : command === "/favorite" - ? (this.options.models ?? []) - : command === "/developer" - ? ["on", "off"] - : command === "/review" - ? ["working", "last-turn", "off"] - : command === "/vim" - ? ["on", "off"] - : ["compact", "full", "focus"]; + ? [ + ...this.providerInventory.flatMap((provider) => + provider.models.map((model) => `${provider.providerId}/${model.modelId}`), + ), + ...(this.providerInventory.length === 0 ? (this.options.models ?? []) : []), + ] + : command === "/providers" || + command === "/login" || + command === "/logout" || + command === "/refresh" + ? this.providerInventory.map((provider) => provider.providerId) + : command === "/thinking" + ? (() => { + const model = this.options.modelCatalog?.find( + (candidate) => candidate.modelId === this.view.model, + ); + return model === undefined ? THINKING_LEVELS : supportedThinkingLevels(model); + })() + : command === "/theme" + ? themeNames(this.themeDefinitions) + : command === "/favorite" + ? (this.options.models ?? []) + : command === "/developer" + ? ["on", "off"] + : command === "/review" + ? ["working", "last-turn", "off"] + : command === "/vim" + ? ["on", "off"] + : ["compact", "full", "focus"]; return values .filter((value) => value.toLowerCase().startsWith(query.toLowerCase())) .map((value) => `${command} ${value}`); @@ -1492,8 +1562,11 @@ export class AxlApp { const overview = projection.overview; const activityChanged = this.liveAssistant.replace(overview.activity); if (this.hydrating) return; + if (overview.activeOperationId !== undefined) this.awaitingOperationOwnership = false; const working = - overview.activeOperationId !== undefined || (this.sending && this.activeRequest !== "turn"); + overview.activeOperationId !== undefined || + this.awaitingOperationOwnership || + (this.sending && this.activeRequest !== "turn"); if (activityChanged || working !== this.view.working) { this.setWorking(working); this.redraw(); @@ -1578,6 +1651,9 @@ export class AxlApp { const completesOperation = event.type === "session.error" || (event.type === "assistant.message" && event.payload.stopReason !== "tool_use"); + if (completesOperation || event.type === "context.compacted") { + this.awaitingOperationOwnership = false; + } if (event.type === "assistant.message") { this.liveAssistant.clear(); } @@ -1819,8 +1895,18 @@ export class AxlApp { } else if (key.kind === "tab") { if (!this.acceptCompletion()) this.editor.apply(key); } else if (key.kind === "escape") { - if (this.view.working) void this.interrupt(); - else if (this.editorMode === "vim") this.vim.handle(key, this.editor); + if (this.providerOperation !== undefined) { + this.providerOperation.abort(); + this.notice = this.view.palette.dim("· provider operation cancelled"); + } else if ( + this.view.working || + this.sending || + this.activeRequest !== undefined || + this.awaitingOperationOwnership || + this.sessionSubscription?.projector.overview.activeOperationId !== undefined + ) { + void this.interrupt(); + } else if (this.editorMode === "vim") this.vim.handle(key, this.editor); else { this.editor.clear(); this.notice = undefined; @@ -2099,6 +2185,11 @@ export class AxlApp { } private handleInterruptKey(): void { + if (this.providerOperation !== undefined) { + this.providerOperation.abort(); + this.notice = this.view.palette.dim("· provider operation cancelled"); + return; + } const now = Date.now(); if (now - this.lastInterrupt < 500) void this.quit(); else { @@ -2338,7 +2429,12 @@ export class AxlApp { return; } if (command === "/favorite") { - this.toggleModelFavorite(argument || this.view.model || this.options.currentModel || ""); + const activeModel = this.view.model ?? this.options.currentModel ?? ""; + const activeProvider = this.view.provider ?? this.options.currentProvider; + this.toggleModelFavorite( + argument || + (activeProvider === undefined ? activeModel : `${activeProvider}/${activeModel}`), + ); return; } if (command === "/developer") { @@ -2414,6 +2510,7 @@ export class AxlApp { this.view.palette.accent("Session"), ` id ${this.sessionId}`, ` profile ${this.view.profile ?? "?"}`, + ` provider ${this.view.provider ?? "?"}`, ` model ${this.view.model ?? "?"}`, ` thinking ${this.view.thinking ?? "?"}`, ...this.requestConfigurationLines(), @@ -2481,7 +2578,19 @@ export class AxlApp { return; } if (command === "/model") { - this.selectModel(argument); + void this.selectModel(argument); + return; + } + if (command === "/providers") { + void this.showProviders(argument || undefined); + return; + } + if (command === "/refresh") { + void this.refreshProviders(argument || undefined); + return; + } + if (command === "/logout") { + void this.logoutProvider(argument || undefined); return; } if (command === "/thinking") { @@ -2491,7 +2600,7 @@ export class AxlApp { if (command === "/login" || command === "/reload" || command === "/compact") { if (this.view.working) this.notice = this.view.palette.dim("· finish or interrupt the turn first"); - else if (command === "/login") void this.openLogin(); + else if (command === "/login") void this.loginProvider(argument || undefined); else if (command === "/reload") void this.reload(); else void this.compact(argument || undefined); return; @@ -2762,7 +2871,7 @@ export class AxlApp { this.notice = this.view.palette.dim("· no active model to favorite"); return; } - if (this.options.models && !this.options.models.includes(modelId)) { + if (this.options.models && !modelId.includes("/") && !this.options.models.includes(modelId)) { this.notice = this.view.palette.error(`✖ unknown model ${modelId}`); return; } @@ -3344,29 +3453,133 @@ export class AxlApp { }); } - private selectModel(modelId: string): void { + private async loadProviderInventory( + providerId?: string, + signal?: AbortSignal, + ): Promise { + const listed = await this.client.listProviders( + providerId === undefined ? {} : { providerId }, + signal === undefined ? {} : { signal }, + ); + if (providerId === undefined) this.providerInventory = listed.providers; + else { + const retained = this.providerInventory.filter( + (provider) => provider.providerId !== providerId, + ); + this.providerInventory = [...retained, ...listed.providers]; + } + this.view.setModels( + this.providerInventory.flatMap((provider) => provider.models.map(formatProviderModel)), + ); + return listed.providers; + } + + private modelSelection( + value: string, + ): { providerId: string; model: ProviderTextModel } | undefined { + const separator = value.indexOf("/"); + if (separator > 0) { + const providerId = value.slice(0, separator); + const modelId = value.slice(separator + 1); + const provider = this.providerInventory.find( + (candidate) => candidate.providerId === providerId, + ); + const model = provider?.models.find((candidate) => candidate.modelId === modelId); + return model === undefined ? undefined : { providerId, model }; + } + const candidates = this.providerInventory.flatMap((provider) => + provider.models + .filter((model) => model.modelId === value) + .map((model) => ({ providerId: provider.providerId, model })), + ); + return ( + candidates.find((candidate) => candidate.providerId === this.view.provider) ?? + (candidates.length === 1 ? candidates[0] : undefined) + ); + } + + private async selectModel(modelId: string): Promise { if (this.view.working) { this.notice = this.view.palette.dim("· finish or interrupt the turn first"); return; } - const models = this.options.models; - if (!models?.length) { - this.notice = this.view.palette.dim("· model selection is unavailable over this attachment"); + try { + await this.loadProviderInventory(); + } catch (error) { + const models = this.options.models; + if (!models?.length) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + this.redraw(); + return; + } + if (modelId) { + if (!models.includes(modelId)) + this.notice = this.view.palette.error(`✖ unknown model ${modelId}`); + else await this.configure({ modelId }); + return; + } + this.openLegacyModelPicker(models); + this.redraw(); return; } + const selections = this.providerInventory.flatMap((provider) => + provider.models.map((model) => ({ provider, model })), + ); if (modelId) { - if (!models.includes(modelId)) { - this.notice = this.view.palette.error(`✖ unknown model ${modelId}`); + const selected = this.modelSelection(modelId); + if (selected === undefined) { + this.notice = this.view.palette.error(`✖ unknown or ambiguous model ${modelId}`); + this.redraw(); return; } - void this.configure({ modelId }); + await this.configure({ providerId: selected.providerId, modelId: selected.model.modelId }); return; } const favorites = new Set(this.modelFavorites); - const ordered = models.toSorted((left, right) => { - const favoriteOrder = Number(favorites.has(right)) - Number(favorites.has(left)); - return favoriteOrder; + const ordered = selections.toSorted((left, right) => { + const leftKey = `${left.provider.providerId}/${left.model.modelId}`; + const rightKey = `${right.provider.providerId}/${right.model.modelId}`; + const favoriteOrder = + Number(favorites.has(rightKey) || favorites.has(right.model.modelId)) - + Number(favorites.has(leftKey) || favorites.has(left.model.modelId)); + return ( + favoriteOrder || + left.provider.displayName.localeCompare(right.provider.displayName) || + left.model.displayName.localeCompare(right.model.displayName) + ); }); + this.openPicker({ + title: "Select model by provider", + items: ordered.map(({ provider, model }) => { + const key = `${provider.providerId}/${model.modelId}`; + const favorite = favorites.has(key) || favorites.has(model.modelId); + const availability = + model.availability.status === "available" + ? "" + : `${model.availability.status}: ${model.availability.reason ?? "not selectable"}`; + return { + value: key, + label: `${favorite ? "◆ " : ""}${provider.displayName} · ${model.displayName}`, + description: [model.apiDialect, availability].filter(Boolean).join(" · "), + }; + }), + current: `${this.view.provider ?? this.options.currentProvider ?? ""}/${this.view.model ?? this.options.currentModel ?? ""}`, + onPick: (value) => { + this.overlays.close(); + const selected = this.modelSelection(value); + if (selected !== undefined) { + void this.configure({ providerId: selected.providerId, modelId: selected.model.modelId }); + } + }, + }); + this.redraw(); + } + + private openLegacyModelPicker(models: readonly string[]): void { + const favorites = new Set(this.modelFavorites); + const ordered = models.toSorted( + (left, right) => Number(favorites.has(right)) - Number(favorites.has(left)), + ); this.openPicker({ title: "Select model", items: ordered.map((id) => ({ @@ -3402,6 +3615,10 @@ export class AxlApp { } private thinkingLevels(): readonly ThinkingLevel[] { + const providerModel = this.providerInventory + .find((provider) => provider.providerId === this.view.provider) + ?.models.find((model) => model.modelId === this.view.model); + if (providerModel !== undefined) return providerModel.supportedThinkingLevels; const model = this.options.modelCatalog?.find( (candidate) => candidate.modelId === this.view.model, ); @@ -4138,6 +4355,271 @@ export class AxlApp { } } + private providerById(providerId: string): ProviderInventoryGroup | undefined { + return this.providerInventory.find((provider) => provider.providerId === providerId); + } + + private chooseProvider( + title: string, + providers: readonly ProviderInventoryGroup[], + onPick: (providerId: string) => void, + ): void { + this.openPicker({ + title, + items: providers.map((provider) => ({ + value: provider.providerId, + label: provider.displayName, + description: `${provider.authentication.phase.replaceAll("_", " ")} · ${provider.models.length} models`, + })), + current: this.view.provider ?? this.options.currentProvider ?? "", + onPick, + }); + } + + private async showProviders(providerId?: string): Promise { + const controller = new AbortController(); + this.providerOperation?.abort(); + this.providerOperation = controller; + this.notice = this.view.palette.dim("· checking provider status · Esc to cancel"); + this.redraw(); + try { + const [providers, statuses] = await Promise.all([ + this.loadProviderInventory(providerId, controller.signal), + this.client.providerAuthenticationStatus(providerId === undefined ? {} : { providerId }, { + signal: controller.signal, + }), + ]); + const statusById = new Map(statuses.providers.map((status) => [status.providerId, status])); + this.commitLines( + providers.flatMap((provider) => { + const status = statusById.get(provider.providerId) ?? provider.authentication; + return [ + this.view.palette.accent(`${provider.displayName} (${provider.providerId})`), + ` authentication ${authenticationLabel(status)}`, + ` catalog ${provider.catalog.refreshable ? "dynamic" : "static"} · ${provider.models.length} text models`, + ...(provider.catalogError === undefined + ? [] + : [ + ` action ${provider.catalogError.action.replaceAll("_", " ")} · ${sanitizeTerminalText(provider.catalogError.message)}`, + ]), + ]; + }), + ); + this.notice = undefined; + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + } finally { + if (this.providerOperation === controller) this.providerOperation = undefined; + this.redraw(); + } + } + + private async refreshProviders(providerId?: string): Promise { + const controller = new AbortController(); + this.providerOperation?.abort(); + this.providerOperation = controller; + this.notice = this.view.palette.dim("· refreshing provider catalogs · Esc to cancel"); + this.redraw(); + try { + const result = await this.client.refreshProviderCatalogs( + providerId === undefined ? {} : { providerId }, + { signal: controller.signal }, + ); + await this.loadProviderInventory(providerId, controller.signal); + this.commitLines( + result.providers.map((provider) => + provider.error === undefined + ? this.view.palette.dim( + `· ${provider.providerId} catalog ${provider.status} · ${provider.modelCount} models`, + ) + : this.view.palette.error( + `✖ ${provider.providerId} · ${sanitizeTerminalText(provider.error.message)} · action: ${provider.error.action.replaceAll("_", " ")}`, + ), + ), + ); + this.notice = undefined; + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + } finally { + if (this.providerOperation === controller) this.providerOperation = undefined; + this.redraw(); + } + } + + private async loginProvider(providerId?: string, method?: ProviderLoginMethod): Promise { + let providers: readonly ProviderInventoryGroup[]; + try { + providers = await this.loadProviderInventory(providerId); + } catch (error) { + if (this.options.loadLogin !== undefined) { + await this.openLogin(); + return; + } + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + this.redraw(); + return; + } + const provider = providerId === undefined ? undefined : this.providerById(providerId); + if (method === undefined) { + const methods = (["oauth", "api_key"] as const).filter((value) => + (provider === undefined ? providers : [provider]).some((entry) => + entry.loginMethods.includes(value), + ), + ); + if (provider !== undefined && methods.length === 1) { + return this.loginProvider(providerId, methods[0]); + } + if (methods.length === 0) { + this.notice = this.view.palette.error("✖ No interactive login methods available"); + this.redraw(); + return; + } + this.openPicker({ + title: "Select authentication method:", + items: methods.map((value) => ({ + value, + label: value === "oauth" ? "Sign in with an account" : "Sign in with an API key", + })), + current: methods[0] ?? "", + onPick: (value) => { + void this.loginProvider(providerId, value as ProviderLoginMethod); + }, + }); + this.redraw(); + return; + } + if (provider === undefined) { + const controller = new AbortController(); + this.providerOperation?.abort(); + this.providerOperation = controller; + this.notice = this.view.palette.dim("· checking provider configuration · Esc to cancel"); + this.redraw(); + let statuses: readonly ProviderAuthenticationStatus[]; + try { + statuses = ( + await this.client.providerAuthenticationStatus( + {}, + { signal: AbortSignal.any([controller.signal, AbortSignal.timeout(15_000)]) }, + ) + ).providers; + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + this.redraw(); + return; + } finally { + if (this.providerOperation === controller) this.providerOperation = undefined; + } + this.notice = undefined; + providers = providers.map((entry) => ({ + ...entry, + authentication: + statuses.find((status) => status.providerId === entry.providerId) ?? entry.authentication, + })); + const candidates = providers + .filter((entry) => entry.loginMethods.includes(method)) + .sort((a, b) => a.displayName.localeCompare(b.displayName)); + this.openPicker({ + title: "Select provider to configure:", + items: candidates.map((entry) => ({ + value: entry.providerId, + label: entry.displayName, + description: + entry.authentication.phase === "authenticated" + ? entry.authentication.method === undefined + ? "✓ configured" + : "✓ stored" + : entry.authentication.phase === "logged_out" + ? "• unconfigured" + : entry.authentication.phase.replaceAll("_", " "), + })), + current: "", + onPick: (value) => { + void this.loginProvider(value, method); + }, + }); + this.redraw(); + return; + } + const selectedMethod = method; + const controller = new AbortController(); + this.providerOperation?.abort(); + this.providerOperation = controller; + this.notice = this.view.palette.dim(`· authenticating ${provider.displayName} · Esc to cancel`); + this.redraw(); + const dialog = new ProviderLoginOverlay({ + title: `Login to ${provider.displayName}`, + palette: () => this.view.palette, + signal: controller.signal, + cancel: () => controller.abort(), + refresh: () => this.redraw(), + }); + this.overlays.replace(dialog); + this.redraw(); + try { + const status = + this.options.loginProvider === undefined + ? await this.client.loginProvider( + { providerId: provider.providerId, method: selectedMethod }, + { signal: controller.signal }, + ) + : await this.options.loginProvider( + provider.providerId, + selectedMethod, + controller.signal, + dialog, + ); + this.notice = this.view.palette.dim( + `· ${provider.displayName} · ${authenticationLabel(status)} · ${provider.catalog.refreshable && provider.models.length === 0 ? `Run /refresh ${provider.providerId} to load models` : "Use /model to select a model"}`, + ); + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + } finally { + if (this.overlays.active === dialog) this.overlays.close(); + if (this.providerOperation === controller) this.providerOperation = undefined; + this.invalidateScreens(); + this.redraw(true); + } + } + + private async logoutProvider(providerId?: string): Promise { + let providers: readonly ProviderInventoryGroup[]; + try { + providers = await this.loadProviderInventory(providerId); + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + this.redraw(); + return; + } + const selectedId = providerId ?? this.view.provider ?? this.options.currentProvider; + const provider = selectedId === undefined ? undefined : this.providerById(selectedId); + if (provider === undefined) { + this.chooseProvider("Logout provider", providers, (value) => { + this.overlays.close(); + void this.logoutProvider(value); + }); + return; + } + const controller = new AbortController(); + this.providerOperation?.abort(); + this.providerOperation = controller; + this.notice = this.view.palette.dim(`· logging out ${provider.displayName} · Esc to cancel`); + this.redraw(); + try { + const status = await this.client.logoutProvider( + { providerId: provider.providerId }, + { signal: controller.signal }, + ); + this.notice = this.view.palette.dim( + `· ${provider.displayName} · ${authenticationLabel(status)}`, + ); + } catch (error) { + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); + } finally { + if (this.providerOperation === controller) this.providerOperation = undefined; + this.redraw(); + } + } + private async openLogin(): Promise { let definition: LoginDialogDefinition | undefined; try { @@ -4172,6 +4654,7 @@ export class AxlApp { } private async persistPreferences(update: { + providerId?: string; modelId?: string; thinkingLevel?: ThinkingLevel; requestSettings?: ModelRequestSettings; @@ -4204,6 +4687,7 @@ export class AxlApp { } private async configure(update: { + providerId?: string; modelId?: string; thinkingLevel?: ThinkingLevel; requestSettings?: ModelRequestSettings; @@ -4226,9 +4710,7 @@ export class AxlApp { if (update.requestSettings !== undefined) this.notice = this.view.palette.dim("· model request settings updated"); } catch (error) { - this.notice = this.view.palette.error( - `✖ ${error instanceof Error ? error.message : "configuration failed"}`, - ); + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); } finally { this.configuring = false; } @@ -4290,8 +4772,16 @@ export class AxlApp { else await this.client.request("session.followUp", params); } catch (error) { const pendingIndex = this.pendingTurnInputs.indexOf(pending); + if (pendingIndex >= 0) this.pendingTurnInputs.splice(pendingIndex, 1); + if ( + pendingIndex >= 0 && + error instanceof AxlClientError && + error.code === "operation_inactive" + ) { + await this.enqueuePrompt(queued, mode === "steer" ? "front" : "back"); + return; + } if (pendingIndex >= 0) { - this.pendingTurnInputs.splice(pendingIndex, 1); this.pendingAttachments.unshift(...queued.attachments); this.editor.setText([queued.text, this.editor.text].filter(Boolean).join("\n\n")); this.notice = this.view.palette.error( @@ -4348,6 +4838,7 @@ export class AxlApp { readonly text: string; readonly attachments: readonly BlobReference[]; }; + this.awaitingOperationOwnership = true; this.setWorking(true); this.view.beginResponse(); this.redraw(); @@ -4361,6 +4852,7 @@ export class AxlApp { ], }); } catch (error) { + this.awaitingOperationOwnership = false; if (this.isConnectionFailure(error)) { const restored = [ queued.text, @@ -4379,9 +4871,7 @@ export class AxlApp { "✖ delivery unknown · prompts restored for review", ); } else { - this.notice = this.view.palette.error( - `✖ ${error instanceof Error ? error.message : "send failed"}`, - ); + this.notice = this.view.palette.error(`✖ ${providerErrorText(error)}`); } break; } @@ -4389,7 +4879,10 @@ export class AxlApp { } finally { this.sending = false; this.activeRequest = undefined; - this.setWorking(this.sessionSubscription?.projector.overview.activeOperationId !== undefined); + this.setWorking( + this.awaitingOperationOwnership || + this.sessionSubscription?.projector.overview.activeOperationId !== undefined, + ); this.redraw(); } } @@ -4410,6 +4903,7 @@ export class AxlApp { private async compact(instructions?: string): Promise { this.sending = true; this.activeRequest = "compaction"; + this.awaitingOperationOwnership = true; this.setWorking(true); this.notice = undefined; this.redraw(); @@ -4420,13 +4914,17 @@ export class AxlApp { }); this.notice = undefined; } catch (error) { + this.awaitingOperationOwnership = false; this.notice = this.view.palette.error( `✖ ${error instanceof Error ? error.message : "compaction failed"}`, ); } finally { this.sending = false; this.activeRequest = undefined; - this.setWorking(this.sessionSubscription?.projector.overview.activeOperationId !== undefined); + this.setWorking( + this.awaitingOperationOwnership || + this.sessionSubscription?.projector.overview.activeOperationId !== undefined, + ); this.redraw(); void this.drainQueue(); } @@ -4455,12 +4953,14 @@ export class AxlApp { this.interrupting = true; try { let result = await this.client.request("session.interrupt", { sessionId: this.sessionId }); - // Working is shown optimistically before session.send installs daemon ownership. - // Preserve an immediate Escape across that short admission window. - while (!result.interrupted && this.sending && !this.stopped) { + // Working is shown optimistically before send or compaction installs daemon ownership. + // Preserve an immediate Escape until admission finishes or a canonical terminal event clears + // the optimistic state. Stopping the app also ends the retry loop. + while (!result.interrupted && this.view.working && !this.stopped) { await new Promise((resolvePromise) => setTimeout(resolvePromise, 25)); result = await this.client.request("session.interrupt", { sessionId: this.sessionId }); } + if (result.interrupted) this.awaitingOperationOwnership = false; } catch { this.notice = this.view.palette.dim("· turn already finished"); this.redraw(); diff --git a/packages/tui/src/dialog.ts b/packages/tui/src/dialog.ts index 995ad52..fab5640 100644 --- a/packages/tui/src/dialog.ts +++ b/packages/tui/src/dialog.ts @@ -3,7 +3,7 @@ // Full-width terminal panel used by selectors, approvals, and login flows. -import { wrapLine } from "./render.ts"; +import { truncateToWidth, wrapLine } from "./render.ts"; import type { Palette } from "./transcript.ts"; export interface DialogInput { @@ -32,9 +32,16 @@ export function renderDialog(input: DialogInput): string[] { return [ border, "", - ...(title ? [` ${palette.accent((palette.bold ?? ((text) => text))(title))}`, ""] : []), + ...(title + ? [ + ` ${palette.accent((palette.bold ?? ((text) => text))(truncateToWidth(title, inner)))}`, + "", + ] + : []), ...content, - ...(footer === undefined ? [] : ["", ` ${palette.dim(footer)}`]), + ...(footer === undefined + ? [] + : ["", ...wrapLine(palette.dim(footer), inner).map((line) => ` ${line}`)]), "", border, ]; diff --git a/packages/tui/src/index.ts b/packages/tui/src/index.ts index bf76035..b260876 100644 --- a/packages/tui/src/index.ts +++ b/packages/tui/src/index.ts @@ -11,18 +11,19 @@ export * from "./dialog.ts"; export * from "./diff-review.ts"; export * from "./editor.ts"; export * from "./editor-frame.ts"; -export * from "./external-editor.ts"; export * from "./extension-ui.ts"; +export * from "./external-editor.ts"; export * from "./fullscreen.ts"; export * from "./fullscreen-input.ts"; export * from "./highlight.ts"; export * from "./input-buffer.ts"; -export * from "./login-dialog.ts"; export * from "./live-assistant.ts"; +export * from "./login-dialog.ts"; export * from "./markdown.ts"; export * from "./media.ts"; export * from "./overlay.ts"; export * from "./picker.ts"; +export * from "./provider-login.ts"; export * from "./render.ts"; export * from "./setup.ts"; export * from "./stack.ts"; diff --git a/packages/tui/src/picker.ts b/packages/tui/src/picker.ts index b298c0d..8b59dd5 100644 --- a/packages/tui/src/picker.ts +++ b/packages/tui/src/picker.ts @@ -55,7 +55,7 @@ export class PickerOverlay implements Overlay { const visible = list.slice(start, start + this.windowSize); const innerWidth = dialogInnerWidth(width); const labelWidth = Math.min( - 22, + Math.max(1, innerWidth - 2), Math.max(1, ...visible.map((item) => visibleWidth(item.label))), ); const itemLine = (item: PickerItem, selected: boolean): string => { @@ -87,7 +87,7 @@ export class PickerOverlay implements Overlay { } cursor(): { row: number; column: number } { - return { row: 1, column: 4 + visibleWidth(this.filter) }; + return { row: this.options.title ? 4 : 2, column: 4 + visibleWidth(this.filter) }; } handleKey(data: string): void { diff --git a/packages/tui/src/provider-login.ts b/packages/tui/src/provider-login.ts new file mode 100644 index 0000000..bb54052 --- /dev/null +++ b/packages/tui/src/provider-login.ts @@ -0,0 +1,168 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import { dialogInnerWidth, renderDialog } from "./dialog.ts"; +import { decodeOneKey } from "./editor.ts"; +import { type PickerItem, PickerOverlay } from "./picker.ts"; +import { sanitizeTerminalText, visibleWidth, wrapLine } from "./render.ts"; +import type { Palette } from "./transcript.ts"; + +export interface ProviderLoginPrompt { + readonly message: string; + readonly placeholder?: string; + readonly mask?: boolean; + readonly allowEmpty?: boolean; + readonly options?: readonly PickerItem[]; + readonly signal?: AbortSignal; +} + +/** Presentation callbacks only, supplied to the trusted host. Never sent over RPC. */ +export interface ProviderLoginPresentation { + prompt(prompt: ProviderLoginPrompt): Promise; + notify(message: string): void; +} + +export class ProviderLoginOverlay implements ProviderLoginPresentation { + private value = ""; + private disposed = false; + private pending: ProviderLoginPrompt | undefined; + private settle: ((value?: string) => void) | undefined; + private picker: PickerOverlay | undefined; + private message = "Waiting for provider…"; + private position: { row: number; column: number } | undefined; + + private readonly options: { + readonly title: string; + readonly palette: () => Palette; + readonly signal: AbortSignal; + readonly cancel: () => void; + readonly refresh: () => void; + }; + + constructor(options: ProviderLoginOverlay["options"]) { + this.options = options; + } + + prompt(prompt: ProviderLoginPrompt): Promise { + if (this.disposed) return Promise.reject(new Error("Login dialog is closed")); + if (this.pending) return Promise.reject(new Error("A login prompt is already active")); + const signal = + prompt.signal === undefined + ? this.options.signal + : AbortSignal.any([this.options.signal, prompt.signal]); + signal.throwIfAborted(); + this.pending = prompt; + this.value = ""; + return new Promise((resolve, reject) => { + const abort = () => this.settle?.(); + this.settle = (value) => { + signal.removeEventListener("abort", abort); + this.pending = undefined; + this.picker = undefined; + this.value = ""; + this.settle = undefined; + if (value === undefined) reject(new DOMException("Login cancelled", "AbortError")); + else resolve(value); + this.options.refresh(); + }; + signal.addEventListener("abort", abort, { once: true }); + if (prompt.options) { + this.picker = new PickerOverlay({ + title: sanitizeTerminalText(prompt.message), + items: prompt.options.map((item) => ({ + ...item, + label: sanitizeTerminalText(item.label), + ...(item.description === undefined + ? {} + : { description: sanitizeTerminalText(item.description) }), + })), + current: prompt.options[0]?.value ?? "", + palette: this.options.palette, + onPick: (value) => this.settle?.(value), + onCancel: this.options.cancel, + }); + } + this.options.refresh(); + }); + } + + notify(message: string): void { + if (this.disposed) return; + this.message = sanitizeTerminalText(message); + this.options.refresh(); + } + + render(width: number): string[] { + if (this.picker) return this.picker.render(width); + const palette = this.options.palette(); + const prompt = this.pending; + const inner = dialogInnerWidth(width); + const rows = prompt + ? [ + ...(this.message === "Waiting for provider…" + ? [] + : this.message.split("\n").flatMap((line) => wrapLine(line, inner))), + ...wrapLine(sanitizeTerminalText(prompt.message), inner), + ...(prompt.placeholder === undefined + ? [] + : wrapLine(palette.dim(sanitizeTerminalText(prompt.placeholder)), inner)), + ...wrapLine(`> ${prompt.mask ? "*".repeat([...this.value].length) : this.value}`, inner), + ] + : this.message.split("\n").flatMap((line) => wrapLine(line, inner)); + this.position = prompt + ? { + row: 4 + rows.length - 1, + column: Math.min(width - 1, 2 + visibleWidth(rows.at(-1) ?? "")), + } + : undefined; + return renderDialog({ + title: this.options.title, + rows, + footer: prompt ? "escape/ctrl+c cancel · enter submit" : "escape/ctrl+c cancel", + width, + palette, + }); + } + + cursor(): { row: number; column: number } | undefined { + return this.picker?.cursor() ?? this.position; + } + + handleKey(data: string): void { + if (this.picker) { + this.picker.handleKey(data); + return; + } + for (let at = 0; at < data.length; ) { + const { key, next } = decodeOneKey(data, at); + at = next; + if ( + key.kind === "escape" || + (key.kind === "ctrl" && (key.char === "c" || key.char === "d")) + ) { + this.options.cancel(); + return; + } + if (!this.pending) continue; + if (key.kind === "enter") { + const value = this.value.trim(); + if (value || this.pending.allowEmpty) this.settle?.(value); + return; + } + if (key.kind === "backspace") { + const segments = [...new Intl.Segmenter().segment(this.value)]; + this.value = this.value.slice(0, segments.at(-1)?.index ?? 0); + } else if (key.kind === "char" && this.value.length < 16_384) { + this.value += sanitizeTerminalText(key.char); + } + } + } + + dispose(): void { + this.disposed = true; + this.options.cancel(); + this.settle?.(); + this.message = ""; + this.value = ""; + } +} diff --git a/packages/tui/src/setup.ts b/packages/tui/src/setup.ts index a92beb6..a9a13b1 100644 --- a/packages/tui/src/setup.ts +++ b/packages/tui/src/setup.ts @@ -8,12 +8,15 @@ import { decodeOneKey } from "./editor.ts"; import { TerminalInputBuffer } from "./input-buffer.ts"; export interface SetupInput { + readonly isTTY?: boolean; + readonly isRaw?: boolean; on(event: "data", listener: (chunk: Buffer | string) => void): unknown; off(event: "data", listener: (chunk: Buffer | string) => void): unknown; setRawMode?(mode: boolean): unknown; } export interface SetupOutput { + readonly isTTY?: boolean; write(data: string): unknown; } @@ -27,6 +30,7 @@ export class SetupAbortedError extends Error { export interface PromptOptions { readonly mask?: boolean; readonly allowEmpty?: boolean; + readonly signal?: AbortSignal; } /** @@ -42,13 +46,21 @@ export function promptLine( options: PromptOptions = {}, ): Promise { output.write(label); + const previousRaw = input.isRaw ?? false; input.setRawMode?.(true); return new Promise((resolve, reject) => { let value = ""; let inputBuffer: TerminalInputBuffer | undefined; + const abort = (): void => { + output.write("\n"); + done(); + reject(new SetupAbortedError()); + }; const done = (): void => { inputBuffer?.dispose(); input.off("data", listener); + options.signal?.removeEventListener("abort", abort); + input.setRawMode?.(previousRaw); }; const finish = (result: string): void => { output.write("\n"); @@ -60,7 +72,10 @@ export function promptLine( while (index < data.length) { const { key, next } = decodeOneKey(data, index); index = next; - if (key.kind === "ctrl" && (key.char === "c" || key.char === "d")) { + if ( + key.kind === "escape" || + (key.kind === "ctrl" && (key.char === "c" || key.char === "d")) + ) { done(); reject(new SetupAbortedError()); return; @@ -94,6 +109,11 @@ export function promptLine( reject(error); }, }); + if (options.signal?.aborted) { + abort(); + return; + } + options.signal?.addEventListener("abort", abort, { once: true }); input.on("data", listener); }); } diff --git a/packages/tui/src/tool-display.ts b/packages/tui/src/tool-display.ts index 8896c66..c78d447 100644 --- a/packages/tui/src/tool-display.ts +++ b/packages/tui/src/tool-display.ts @@ -30,10 +30,14 @@ const INPUT_PREVIEW_ROWS = 8; const FULL_INPUT_PREVIEW_ROWS = 40; function previewText(text: string, limit: number): string { - const safe = sanitizeTerminalText(text); - if (safe.length <= limit) return safe; + if (text.length <= limit) return sanitizeTerminalText(text); const half = Math.floor(limit / 2); - return `${safe.slice(0, half)}\n…\n${safe.slice(-half)}`; + // Omitted text is never rendered, so sanitize only the bounded visible ends. + // Include a small margin because removed control sequences consume source bytes. + const margin = 256; + const start = sanitizeTerminalText(text.slice(0, half + margin)).slice(0, half); + const end = sanitizeTerminalText(text.slice(-half - margin)).slice(-half); + return `${start}\n…\n${end}`; } function inputPreview( diff --git a/packages/tui/src/tool-transaction.ts b/packages/tui/src/tool-transaction.ts index 7975905..52726c6 100644 --- a/packages/tui/src/tool-transaction.ts +++ b/packages/tui/src/tool-transaction.ts @@ -280,7 +280,15 @@ export class ToolTransactionStore implements Component { `${groupId}:header`, ); } - for (const component of components) append(component.render(width), component.sourceId); + for (const component of components) { + const rendered = component.render(width); + append( + !compact && components.length > 1 + ? rendered.filter((line) => line.trim().length > 0) + : rendered, + component.sourceId, + ); + } append( [ palette.dim( diff --git a/packages/tui/src/transcript.ts b/packages/tui/src/transcript.ts index c5fb525..b139639 100644 --- a/packages/tui/src/transcript.ts +++ b/packages/tui/src/transcript.ts @@ -107,8 +107,9 @@ export type BlobRenderer = ( export class SessionView { palette: Palette; private width: number; - private readonly models: readonly ClientModelInfo[]; + private models: readonly ClientModelInfo[]; private readonly renderBlob: BlobRenderer | undefined; + provider: string | undefined; model: string | undefined; thinking: string | undefined; profile: string | undefined; @@ -124,6 +125,17 @@ export class SessionView { contextTokens: number | undefined = 0; cacheHitPercent: number | undefined; totalCostUsd = 0; + private fallbackCostUsd = 0; + private lastUsage: + | { + readonly inputTokens: number; + readonly outputTokens: number; + readonly cacheReadTokens: number; + readonly cacheWriteTokens: number; + readonly reasoningTokens?: number; + readonly costUsd?: number; + } + | undefined; tokensPerSecond: number | undefined; elapsedSeconds = 0; private responseStartedAt: number | undefined; @@ -147,6 +159,10 @@ export class SessionView { this.width = Math.max(1, width); } + setModels(models: readonly ClientModelInfo[]): void { + this.models = models; + } + cycleThinkingDisplay(): ThinkingDisplay { this.thinkingDisplay = this.thinkingDisplay === "compact" @@ -176,18 +192,53 @@ export class SessionView { } usageLabel(): string { + const total = this.formatUsage( + { + inputTokens: this.inputTokens, + outputTokens: this.outputTokens, + cacheReadTokens: this.cacheReadTokens, + cacheWriteTokens: this.cacheWriteTokens, + costUsd: this.totalCostUsd, + }, + true, + ); + return this.lastUsage === undefined + ? total + : `turn ${this.formatUsage(this.lastUsage)} · total ${total}`; + } + + private formatUsage( + usage: { + readonly inputTokens: number; + readonly outputTokens: number; + readonly cacheReadTokens: number; + readonly cacheWriteTokens: number; + readonly reasoningTokens?: number; + readonly costUsd?: number; + }, + includeContext = false, + ): string { const parts: string[] = []; - if (this.inputTokens) parts.push(`↑${compactNumber(this.inputTokens)}`); - if (this.outputTokens) parts.push(`↓${compactNumber(this.outputTokens)}`); - if (this.cacheReadTokens) parts.push(`R${compactNumber(this.cacheReadTokens)}`); - if (this.cacheWriteTokens) parts.push(`W${compactNumber(this.cacheWriteTokens)}`); - if ((this.cacheReadTokens || this.cacheWriteTokens) && this.cacheHitPercent !== undefined) { + if (usage.inputTokens) parts.push(`↑${compactNumber(usage.inputTokens)}`); + if (usage.outputTokens) parts.push(`↓${compactNumber(usage.outputTokens)}`); + if (usage.cacheReadTokens) parts.push(`R${compactNumber(usage.cacheReadTokens)}`); + if (usage.cacheWriteTokens) parts.push(`W${compactNumber(usage.cacheWriteTokens)}`); + if (usage.reasoningTokens) parts.push(`∴${compactNumber(usage.reasoningTokens)}`); + if ((usage.cacheReadTokens || usage.cacheWriteTokens) && this.cacheHitPercent !== undefined) { parts.push(`CH${this.cacheHitPercent.toFixed(1)}%`); } - if (this.totalCostUsd) parts.push(`$${this.totalCostUsd.toFixed(3)}`); + if (usage.costUsd) parts.push(`$${usage.costUsd.toFixed(3)}`); - const model = this.models.find((candidate) => candidate.modelId === this.model); - if (model === undefined) { + const model = includeContext + ? this.models.find( + (candidate) => + candidate.modelId === this.model && + (candidate.providerId === undefined || candidate.providerId === this.provider), + ) + : undefined; + if (!includeContext) { + if (parts.length === 0) parts.push("no usage"); + } else if (model === undefined) { if (parts.length === 0) parts.push("ready"); } else { const percent = @@ -223,10 +274,12 @@ export class SessionView { /** Renders an event already reduced by the shared SDK subscription projector. */ present(event: CanonicalEvent): readonly string[] { + const previousProvider = this.provider; const previousModel = this.model; const previousThinking = this.thinking; const previousSandbox = this.sandbox; const projected = this.projection.state; + this.provider = projected.provider; this.model = projected.model; this.thinking = projected.thinking; this.profile = projected.profile; @@ -241,7 +294,7 @@ export class SessionView { this.cacheReadTokens = projected.usage.cacheReadTokens; this.cacheWriteTokens = projected.usage.cacheWriteTokens; this.totalTokens = projected.usage.inputTokens + projected.usage.outputTokens; - this.totalCostUsd = projected.usage.costUsd; + this.totalCostUsd = projected.usage.costUsd + this.fallbackCostUsd; const { dim, error } = this.palette; switch (event.type) { case "session.created": @@ -279,15 +332,28 @@ export class SessionView { this.contextTokens = promptTokens; this.cacheHitPercent = promptTokens > 0 ? (usage.cacheReadTokens / promptTokens) * 100 : undefined; - const cost = this.models.find((candidate) => candidate.modelId === this.model)?.cost; - if (usage.costUsd === undefined && cost !== undefined) { - this.totalCostUsd += - (cost.inputUsdPerMTok * usage.inputTokens + - cost.outputUsdPerMTok * usage.outputTokens + - (cost.cacheReadUsdPerMTok ?? 0) * usage.cacheReadTokens + - (cost.cacheWriteUsdPerMTok ?? 0) * usage.cacheWriteTokens) / - 1_000_000; + const cost = this.models.find( + (candidate) => + candidate.modelId === this.model && + (candidate.providerId === undefined || candidate.providerId === this.provider), + )?.cost; + const computedCost = + usage.costUsd ?? + (cost === undefined + ? undefined + : (cost.inputUsdPerMTok * usage.inputTokens + + cost.outputUsdPerMTok * usage.outputTokens + + (cost.cacheReadUsdPerMTok ?? 0) * usage.cacheReadTokens + + (cost.cacheWriteUsdPerMTok ?? 0) * usage.cacheWriteTokens) / + 1_000_000); + if (usage.costUsd === undefined && computedCost !== undefined) { + this.fallbackCostUsd += computedCost; + this.totalCostUsd += computedCost; } + this.lastUsage = { + ...usage, + ...(computedCost === undefined ? {} : { costUsd: computedCost }), + }; if (this.responseStartedAt !== undefined && usage.outputTokens > 0) { const elapsedMs = performance.now() - this.responseStartedAt; this.tokensPerSecond = @@ -344,10 +410,17 @@ export class SessionView { ), ); case "session.error": - return this.errorLines( - sanitizeTerminalText(event.payload.message), - sanitizeTerminalText(event.payload.code), - ); + return [ + ...this.errorLines( + sanitizeTerminalText(event.payload.message), + sanitizeTerminalText(event.payload.code), + ), + ...(event.payload.retryable ? this.wrap(dim(" Action: retry the request.")) : []), + ]; + case "config.provider": + return previousProvider === undefined || previousProvider === this.provider + ? [] + : this.wrap(dim(`· provider ${previousProvider} → ${this.provider}`)); case "config.model": return previousModel === undefined || previousModel === this.model ? [] @@ -419,7 +492,7 @@ export class SessionView { : queued ? `idle +${queued}` : "idle"; - const full = `${activity} · session ${sessionId.slice(0, 8)} · profile ${this.profile ?? "?"} · model ${this.model ?? "?"} · thinking ${this.thinking ?? "?"} · sandbox ${this.sandbox ?? "none"}`; + const full = `${activity} · session ${sessionId.slice(0, 8)} · profile ${this.profile ?? "?"} · provider ${this.provider ?? "?"} · model ${this.model ?? "?"} · thinking ${this.thinking ?? "?"} · sandbox ${this.sandbox ?? "none"}`; return this.palette.dim(truncateToWidth(full, this.width, "")); } diff --git a/packages/tui/test/app.test.ts b/packages/tui/test/app.test.ts index 933db4a..156c3aa 100644 --- a/packages/tui/test/app.test.ts +++ b/packages/tui/test/app.test.ts @@ -12,7 +12,11 @@ import { join } from "node:path"; import { PassThrough as NodePassThrough } from "node:stream"; import test, { type TestContext } from "node:test"; -import { AxlDaemon, type SessionInteractionRequest } from "@axl/daemon"; +import { + AxlDaemon, + type ProviderManagementService, + type SessionInteractionRequest, +} from "@axl/daemon"; import type { TerminalExtension } from "@axl/extension-api"; import { type CompactionSettings, @@ -20,7 +24,6 @@ import { type ModelTurnRequest, ToolRegistry, } from "@axl/kernel"; -import { DEFAULT_MODEL_REQUEST_SETTINGS } from "@axl/protocol"; import type { CanonicalEvent, EventPayloadMap, @@ -29,6 +32,7 @@ import type { SessionId, Usage, } from "@axl/protocol"; +import { DEFAULT_MODEL_REQUEST_SETTINGS } from "@axl/protocol"; import { subscribeSession } from "@axl/sdk"; import { connectUnixClient, createUnixDaemonHost } from "@axl/sdk/unix"; @@ -70,13 +74,14 @@ async function startStack( ) => ToolRegistry = () => new ToolRegistry(), sandbox?: EventPayloadMap["sandbox.configured"], compaction?: Partial, + providerManagement?: ProviderManagementService, ) { const directory = await mkdtemp(join(tmpdir(), "axl-tui-")); - context.after(() => rm(directory, { recursive: true, force: true })); const socketPath = join(directory, "axl.sock"); const daemon = new AxlDaemon({ socketPath, dataDirectory: join(directory, "data"), + ...(providerManagement === undefined ? {} : { providerManagement }), runtime: ({ selection, interact }) => ({ model, configRequest: selection.requestSettings ?? DEFAULT_MODEL_REQUEST_SETTINGS, @@ -84,6 +89,9 @@ async function startStack( system: "You are Axl.", ...(sandbox === undefined ? {} : { sandbox }), ...(compaction === undefined ? {} : { compaction }), + ...(selection.providerId === undefined + ? {} + : { configProvider: { providerId: selection.providerId } }), ...(selection.modelId === undefined ? {} : { configModel: { modelId: selection.modelId } }), ...(selection.thinkingLevel === undefined ? {} @@ -97,7 +105,10 @@ async function startStack( }), }); await daemon.start(); - context.after(() => daemon.stop()); + context.after(async () => { + await daemon.stop(); + await rm(directory, { recursive: true, force: true }); + }); return { socketPath, directory: await realpath(directory) }; } @@ -154,6 +165,7 @@ test("a full round trip: type, send, render the reply, detach, resume", async (c input.write("hello axl\r"); await until(() => text().includes("↑1 ↓1"), "canonical assistant reply"); + await until(() => text().includes("tok/s"), "throughput repaint"); assert.match(text(), /│ hello axl/); assert.match(text(), /the answer/); assert.match(text(), /↑1 ↓1/); @@ -980,6 +992,7 @@ test("terminal resize coalesces bursts and leaves one live frame", async (contex const beforeNextTurn = text().length; input.write("after resize\r"); await until(() => text().includes("↑2 ↓2"), "canonical post-resize reply"); + await new Promise((resolve) => setTimeout(resolve, 30)); assert.match(text().slice(beforeNextTurn), /after resize/); const terminal = new VirtualTerminal(100, 30); terminal.write(`${latestResizeOutput}${text().slice(beforeNextTurn)}`); @@ -1127,9 +1140,11 @@ test("Ctrl+V paste, Shift+Enter, and searchable hotkeys behave", async (context) }); test("Escape interrupts a running operation", async (context) => { + let operationStarted = false; let operationAborted = false; const blockingPort: ModelPort = { stream(request) { + operationStarted = true; return (async function* (): AsyncGenerator { await new Promise((resolve) => { if (request.signal?.aborted) resolve(); @@ -1153,7 +1168,7 @@ test("Escape interrupts a running operation", async (context) => { await until(() => text().includes("\x1b[>4;2m"), "keyboard negotiation"); input.write("start work\r"); - await until(() => text().includes("Working"), "working state"); + await until(() => text().includes("Working") && operationStarted, "running model operation"); input.write("\x1b[27u"); await until(() => operationAborted, "escape interruption"); app.stop(); @@ -1769,6 +1784,7 @@ test("/model opens a selector and switches the model live", async (context) => { cwd: directory, color: false, models: ["gpt-5", "gpt-4.1", "gpt-4o-mini"], + currentProvider: "test-provider", currentModel: "gpt-5", onModelChange: (modelId) => switched.push(modelId), onPreferenceChange: (update) => { @@ -1791,6 +1807,180 @@ test("/model opens a selector and switches the model live", async (context) => { app.stop(); }); +test("provider commands group models, show status, mutate auth, and cancel refresh", async (context) => { + const calls: string[] = []; + let blockRefresh = false; + let refreshCancelled = false; + const provider = ( + providerId: string, + displayName: string, + availability: "available" | "unavailable", + ) => ({ + providerId, + displayName, + enabled: true, + authMethods: ["environment" as const], + loginMethods: ["api_key" as const], + authentication: { providerId, phase: "idle" as const }, + catalog: { refreshable: true }, + models: [ + { + providerId, + modelId: "shared-model", + displayName: `${displayName} Model`, + apiDialect: "openai-chat", + capabilities: { toolUse: true, structuredOutput: true, imageInput: false }, + reasoning: false, + supportedThinkingLevels: ["off" as const], + contextWindow: 16_000, + maxOutputTokens: 2_000, + availability: { + status: availability, + ...(availability === "unavailable" ? { reason: "region is not configured" } : {}), + }, + }, + ], + }); + const providers = [ + provider("alpha", "Alpha", "available"), + provider("beta", "Beta", "unavailable"), + ]; + const service: ProviderManagementService = { + list: (params) => + Promise.resolve({ + providers: + params.providerId === undefined + ? providers + : providers.filter((entry) => entry.providerId === params.providerId), + }), + refresh: async (params, signal) => { + calls.push(`refresh:${params.providerId ?? "all"}`); + if (blockRefresh) { + await new Promise((resolvePromise) => { + signal?.addEventListener( + "abort", + () => { + refreshCancelled = true; + resolvePromise(); + }, + { once: true }, + ); + }); + signal?.throwIfAborted(); + } + return { + providers: [ + { providerId: params.providerId ?? "alpha", status: "refreshed", modelCount: 1 }, + ], + }; + }, + authenticationStatus: (params) => + Promise.resolve({ + providers: providers + .filter( + (entry) => params.providerId === undefined || entry.providerId === params.providerId, + ) + .map((entry) => ({ + providerId: entry.providerId, + phase: "authenticated" as const, + source: "test environment", + })), + }), + login: (params) => { + calls.push(`login:${params.providerId}:${params.method}`); + return Promise.resolve({ providerId: params.providerId, phase: "authenticated" }); + }, + logout: (params) => { + calls.push(`logout:${params.providerId}`); + return Promise.resolve({ providerId: params.providerId, phase: "logged_out" }); + }, + }; + const { socketPath, directory } = await startStack( + context, + port, + () => new ToolRegistry(), + undefined, + undefined, + service, + ); + const input = new PassThrough(); + const { output, text } = captureOutput(); + const preferences: Array> = []; + const app = await AxlApp.start({ + client: await connectUnixClient(socketPath), + input, + output, + cwd: directory, + color: false, + currentProvider: "alpha", + currentModel: "shared-model", + loginProvider: async (providerId, method, _signal, presentation) => { + calls.push(`host-login:${providerId}:${method}`); + assert.equal( + await presentation.prompt({ + message: "Enterprise domain (blank for default)", + allowEmpty: true, + }), + "", + ); + assert.equal( + await presentation.prompt({ message: "Provider secret", mask: true }), + "runtime-login-secret", + ); + return { providerId, phase: "authenticated", method }; + }, + onPreferenceChange: (update) => { + preferences.push(update); + }, + }); + + input.write("/model\r"); + await until(() => text().includes("Select model by provider"), "grouped model selector"); + assert.match(text(), /Alpha · Alpha Model/); + assert.match(text(), /Beta · Beta Model/); + assert.match(text(), /unavailable: region is not configured/); + input.write("\x1b"); + await new Promise((resolve) => setTimeout(resolve, 30)); + + input.write("/model beta/shared-model\r"); + await until( + () => preferences.some((value) => value.providerId === "beta"), + "provider model switch", + ); + assert.deepEqual(preferences.at(-1), { providerId: "beta", modelId: "shared-model" }); + + input.write("/providers beta\r"); + await until(() => text().includes("test environment"), "provider status"); + input.write("/logout beta\r"); + await until(() => calls.includes("logout:beta"), "provider logout"); + input.write("/login\r"); + await until(() => text().includes("Select authentication method:"), "login method selector"); + input.write("\r"); + await until(() => text().includes("Select provider to configure:"), "login provider selector"); + input.write("beta\r"); + await until( + () => text().includes("Enterprise domain (blank for default)"), + "inline login prompt", + ); + input.write("\r"); + await until(() => text().includes("Provider secret"), "secret prompt"); + input.write("runtime-login-secret\r"); + await until(() => calls.includes("host-login:beta:api_key"), "provider login"); + await new Promise((resolve) => setTimeout(resolve, 30)); + assert.equal(calls.includes("login:beta:api_key"), false); + + blockRefresh = true; + input.write("/refresh beta\r"); + await until( + () => calls.filter((value) => value === "refresh:beta").length === 1, + "catalog refresh", + ); + input.write("\x1b"); + await until(() => refreshCancelled, "catalog refresh cancellation"); + assert.doesNotMatch(text(), /runtime-login-secret/); + app.stop(); +}); + test("/theme previews message and tool surfaces live", async (context) => { const { socketPath, directory } = await startStack(context); const input = new PassThrough(); @@ -1880,6 +2070,7 @@ test("/model digit selection and Esc cancel behave", async (context) => { cwd: directory, color: false, models: ["gpt-5", "gpt-4o-mini"], + currentProvider: "test-provider", onModelChange: (modelId) => switched.push(modelId), }); @@ -1948,7 +2139,6 @@ test("/login renders a provider-neutral injected dialog", async (context) => { test("/reload requests a runtime rebuild and renders the boundary", async (context) => { const directory = await mkdtemp(join(tmpdir(), "axl-tui-")); - context.after(() => rm(directory, { recursive: true, force: true })); const socketPath = join(directory, "axl.sock"); const daemon = new AxlDaemon({ socketPath, @@ -1968,7 +2158,10 @@ test("/reload requests a runtime rebuild and renders the boundary", async (conte }), }); await daemon.start(); - context.after(() => daemon.stop()); + context.after(async () => { + await daemon.stop(); + await rm(directory, { recursive: true, force: true }); + }); const input = new PassThrough(); const { output, text } = captureOutput(); @@ -2444,6 +2637,7 @@ test("request settings are visible, configurable, persisted, and survive resume" output, cwd: directory, color: false, + currentProvider: "test-provider", currentModel: "test-model", onPreferenceChange: (update) => { preferences.push(update); diff --git a/packages/tui/test/provider-login.test.ts b/packages/tui/test/provider-login.test.ts new file mode 100644 index 0000000..7e22d5b --- /dev/null +++ b/packages/tui/test/provider-login.test.ts @@ -0,0 +1,71 @@ +// SPDX-FileCopyrightText: 2026 Hari Srinivasan +// SPDX-License-Identifier: Apache-2.0 + +import assert from "node:assert/strict"; +import test from "node:test"; + +import { PLAIN_PALETTE, ProviderLoginOverlay, visibleWidth } from "../src/index.ts"; + +function fixture() { + const controller = new AbortController(); + const overlay = new ProviderLoginOverlay({ + title: "Login to GitHub Copilot", + palette: () => PLAIN_PALETTE, + signal: controller.signal, + cancel: () => controller.abort(), + refresh: () => {}, + }); + return { overlay, controller }; +} + +test("inline login accepts blank text, masks secrets, and places the cursor after wrapped input", async () => { + const { overlay } = fixture(); + const domain = overlay.prompt({ + message: "GitHub Enterprise domain (blank for github.com)", + placeholder: "company.ghe.com", + allowEmpty: true, + }); + const rendered = overlay.render(60); + assert.match(rendered.join("\n"), /company.ghe.com\n {2}> /); + overlay.handleKey("\r"); + assert.equal(await domain, ""); + const secret = overlay.prompt({ message: "API key", mask: true }); + overlay.handleKey("obviously-fake-secret"); + const rows = overlay.render(30); + assert.doesNotMatch(rows.join("\n"), /obviously-fake-secret/); + assert.match(rows.join("\n"), /\*{21}/); + assert.equal( + rows.every((row) => visibleWidth(row) <= 30), + true, + ); + const cursor = overlay.cursor(); + assert.ok(cursor); + assert.equal(cursor.column, visibleWidth(rows[cursor.row] ?? "")); + overlay.handleKey("\r"); + assert.equal(await secret, "obviously-fake-secret"); + assert.doesNotMatch(overlay.render(60).join("\n"), /\*{21}|obviously-fake-secret/); + overlay.dispose(); +}); + +test("inline login retains authorization instructions and cancels pending prompts", async () => { + const { overlay, controller } = fixture(); + overlay.notify("Open https://example.com/authorize\nCode: EXAMPLE"); + const prompt = overlay.prompt({ message: "Paste code", mask: true }); + assert.match(overlay.render(60).join("\n"), /https:\/\/example.com\/authorize/); + controller.abort(); + await assert.rejects(prompt, /Login cancelled/); + overlay.dispose(); + const next = fixture(); + const selected = next.overlay.prompt({ + message: "Choose account", + options: [ + { value: "one", label: "First" }, + { value: "two", label: "Second" }, + ], + }); + next.overlay.handleKey("\x1b[B\r"); + assert.equal(await selected, "two"); + const cancelled = next.overlay.prompt({ message: "Account" }); + next.overlay.handleKey("\x1b"); + await assert.rejects(cancelled, /Login cancelled/); +}); diff --git a/packages/tui/test/transcript.test.ts b/packages/tui/test/transcript.test.ts index 460fc32..05526aa 100644 --- a/packages/tui/test/transcript.test.ts +++ b/packages/tui/test/transcript.test.ts @@ -8,9 +8,6 @@ import assert from "node:assert/strict"; import { readFile } from "node:fs/promises"; import test from "node:test"; - -import { ConversationProjector } from "@axl/sdk"; - import { type CanonicalEvent, EVENT_FORMAT_VERSION, @@ -20,6 +17,7 @@ import { parseEventId, parseSessionId, } from "@axl/protocol"; +import { ConversationProjector } from "@axl/sdk"; import { PLAIN_PALETTE, SessionView } from "../src/index.ts"; @@ -203,11 +201,15 @@ test("reports cumulative usage, cache hit rate, cost, and local throughput", () outputTokens: 5, cacheReadTokens: 20, cacheWriteTokens: 2, + reasoningTokens: 3, costUsd: 0.125, }, }), ); - assert.equal(view.usageLabel(), "↑10 ↓5 R20 W2 CH62.5% $0.125"); + assert.equal( + view.usageLabel(), + "turn ↑10 ↓5 R20 W2 ∴3 CH62.5% $0.125 · total ↑10 ↓5 R20 W2 CH62.5% $0.125", + ); assert.match(view.tpsLabel(), /tok\/s$/); }); diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 571a39c..fc87372 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -23,9 +23,27 @@ importers: packages/ai: dependencies: + '@aws-sdk/credential-provider-node': + specifier: 3.972.82 + version: 3.972.82 '@axl/protocol': specifier: workspace:* version: link:../protocol + '@azure/identity': + specifier: 4.13.2 + version: 4.13.2 + '@smithy/hash-node': + specifier: 4.5.2 + version: 4.5.2 + '@smithy/protocol-http': + specifier: 5.6.2 + version: 5.6.2 + '@smithy/signature-v4': + specifier: 5.7.3 + version: 5.7.3 + google-auth-library: + specifier: 11.0.2 + version: 11.0.2 undici: specifier: 8.10.2 version: 8.10.2 @@ -208,6 +226,118 @@ importers: packages: + '@aws-sdk/core@3.977.9': + resolution: {integrity: sha512-reqPFEQrZxDZpeGj4PFMepBeR5LGYHRqq/L0motTzgFkCRBA4rFdaVXDSLYyGHhxVz7sT2PDnPN9CluGSfgyJA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-env@3.972.70': + resolution: {integrity: sha512-H404B7dJl2mCrBqahDEYsanB0xhdDp6tXnXcTUnXmmpy2Q3J0Ho0bUajZ2jr/RdwzCyS59Gi8xXIFwPLGBl6Uw==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-http@3.972.72': + resolution: {integrity: sha512-X98zYOrVOeuosCX+6ktf29FC2N2GHPLia7qv6mzPzTc+RPAuHWCDS++Z6JK7eGYqb/v6uaW7bAXaOvDBfol+0w==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-ini@3.973.15': + resolution: {integrity: sha512-Rykg6s5ceBuynMOGWgoowO4N+27JfnqXAnVaSunZl0hOO1XodSrxGNz6sCEbnmS0lAfQZDKyb3fbr46gSuv6Sg==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-login@3.972.77': + resolution: {integrity: sha512-Jb59xfEISoN5mmbnA+HYqdtrSX3CgCtJoof+V5D8/TgUI56W63GEEd5Y58WijU3Ou6+WEgaLD1feVzaRXV5IDQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-node@3.972.82': + resolution: {integrity: sha512-znDkEOGXB8W3kG1LJUKP3foBZY/9qLM0eil/DxWXSp37XsdsRLQHE/d/OaCGGVgKpA6znR38h/+INk8do1FjiA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-process@3.972.70': + resolution: {integrity: sha512-2ry03fGRJr4sV3jI+ocjj5JqALnFD6ymM5KiNCDZMvq8bX2GSbE0vji4aM43TVCl2nXqqLRZaUxdq/KeWRAY4Q==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-sso@3.973.14': + resolution: {integrity: sha512-jkhg/8ocAAoc0RFyLMhCw+/zZh7gystQgd4F4hznNa8P4Cc501PQmxd+jGLiMHodPJ+7Zv/3znM62gZojyasmA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/credential-provider-web-identity@3.972.76': + resolution: {integrity: sha512-d3AGyVu759PGr35mEB2s22xxlNEA5rpdxtSPJthfPFJvoQ8dt357iVPECqWfUxXp1toJAvKmbtcIYVGigaGsCA==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/nested-clients@3.997.44': + resolution: {integrity: sha512-NhEgryjlBF9w38ZXqGymQV28IhkYa1mKhlbYnqIis57AYwWGVYfUPgg/qC2rLRqOUfblxx++irvju10kVTa8Vw==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/signature-v4-multi-region@3.996.46': + resolution: {integrity: sha512-L+2xZTye/2T96f3lwCws0Zw6GG2JHZW9e8FpVgGBeeExSKyeoZ6CWRpBml/7DNiK/O26jrgPM9F+Ay8VkgzUWQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/token-providers@3.1116.0': + resolution: {integrity: sha512-ygIivKqh8aHzNkucOCXHyIBgBpLPfrSI0mCqXF+vLBsPTUKqj0VSqAY0GFPe7lQl4HntjOcQ+KSyS7oUV2C54Q==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/types@3.974.5': + resolution: {integrity: sha512-LkwLL2BLbC6wNNm4JaH9mbEqBMdOZCct6VAYqhdN4U1xrWM+fUJQEfbHwQgDypapOWTRtlk25akb5afM0P8CIQ==} + engines: {node: '>=20.0.0'} + + '@aws-sdk/xml-builder@3.972.40': + resolution: {integrity: sha512-wlFmCIGUlwF4zx/kncw+bmxTQh1HeSJq4mYV/V5cZUSJadDP3kXvGW8Rn21cimj/7y9ju+47oYWXi97vF7czaA==} + engines: {node: '>=20.0.0'} + + '@aws/lambda-invoke-store@0.3.0': + resolution: {integrity: sha512-sl4Bm6yiMNYrZKkqqDFWN0UfnWhlS8ivKxrYl+6t0gCLrqr8y3B2IqZZbFRkfaVVp7C/baApyh71P+LeE1A2sQ==} + engines: {node: '>=18.0.0'} + + '@azure/abort-controller@2.2.0': + resolution: {integrity: sha512-fNAjWnA/nZ2jz31kxR/AqRaUT8ewHBw/WuBIosK0moMy1C9e5ValbDfFdIxJzVOOYaYkV/b2F1S4H/aHiqfVQg==} + engines: {node: '>=22.0.0'} + + '@azure/core-auth@1.11.0': + resolution: {integrity: sha512-IUZydyTUkDnYdstOW9pFOOUQlBjAepK5teihDE3x6yxsPJs/hsAaaYpeGxdxrgtOiJbBKSjKW7MDk7AEhb4LRg==} + engines: {node: '>=22.0.0'} + + '@azure/core-client@1.11.1': + resolution: {integrity: sha512-2QygG2F76ZpMP2eMztiJvAiFMu71M9rDeU7vO/QKg5Css7MgM4frUOslFjhVjRhbGaCNPtz/S8M6y46/fFKVuQ==} + engines: {node: '>=22.0.0'} + + '@azure/core-process@1.0.0': + resolution: {integrity: sha512-/shnJ+ooO8WPxDhPEeI/2oRQuubn16gZ6CvlbpWbEswZfzwI9tI/sMAHmF3x1LuQ9yZYXfLW3TjzGMLEC5blKg==} + engines: {node: '>=22.0.0'} + + '@azure/core-rest-pipeline@1.25.0': + resolution: {integrity: sha512-bMs8ekJLjX8wPV+9IPBges1SLPyuDtE9g5gLDWOpxzKcoOFQnpLGkbcT1tdw3FaAmDS1gnPmMmJ6y/T5B96kIA==} + engines: {node: '>=22.0.0'} + + '@azure/core-tracing@1.4.0': + resolution: {integrity: sha512-eGwxD0AtncrxeBM4tG8R55Pc3rdX1hNW2WibJAgYpCVA6E93mvvVH+LcssoVjOBrSKWS55yEIHsk0X8ctHmfOQ==} + engines: {node: '>=22.0.0'} + + '@azure/core-util@1.14.0': + resolution: {integrity: sha512-9n2pWK61veAuN0V20t9lOuoV4CFMdyAZ1ygZzvBGk/pBBJRib/PjL9PLXa/aI2CcPpyHfqVsxxqLCYl6uZlfDw==} + engines: {node: '>=22.0.0'} + + '@azure/identity@4.13.2': + resolution: {integrity: sha512-NXL2/pCJctLxgw8bvrwwgge743kEq8LBT+O1pmV0vyUwetzFPH9auP6jhkU/cgZCPPtWoewAe3ncaGCgPo07fA==} + engines: {node: '>=22.0.0'} + + '@azure/logger@1.4.0': + resolution: {integrity: sha512-rbAE25KUfjU/s3XHUdJgceoCP5dEOpMx85J04kF+QMdta73XkuG9JGHHinch+XIoKpBdqljin+KqURpJriSzLA==} + engines: {node: '>=22.0.0'} + + '@azure/msal-browser@5.21.0': + resolution: {integrity: sha512-80OcuXDErmcEDAIH9pBtSqBsed2sPT/IWmbG3xHLoPMl5zc8TINd6SlJAbVSmN5huGa3xGAg5qR7VnpaIEK0Zw==} + engines: {node: '>=0.8.0'} + + '@azure/msal-common@16.13.0': + resolution: {integrity: sha512-rOAy0KUcyBbdwVJ+f3uPpthXatFLLZN+/KWAsTLzk1aB23Xl9DRmmXYwSvBFOZyXj4jUQQ5FKxxRkhAFW1fOow==} + engines: {node: '>=0.8.0'} + + '@azure/msal-common@16.14.0': + resolution: {integrity: sha512-A4rb55hI86Q9tBl/+jBj7TMz7iX2RFgQs/nExFzcAtoI/BFRVdaH5SL/MivrYD7qvweMpN8AgVvVMHV8UBYxew==} + engines: {node: '>=0.8.0'} + + '@azure/msal-node@5.6.0': + resolution: {integrity: sha512-uFY9NxrWHw8PwZx7gAX6PDn+9vdfS05+levc/kwkx77IkjfaldnQbbcQzzDIZ5Hq5Zdr6/z92oAIoRWKp6MnOA==} + engines: {node: '>=20'} + '@biomejs/biome@2.3.5': resolution: {integrity: sha512-HvLhNlIlBIbAV77VysRIBEwp55oM/QAjQEin74QQX9Xb259/XP/D5AGGnZMOyF1el4zcvlNYYR3AyTMUV3ILhg==} engines: {node: '>=14.21.3'} @@ -453,13 +583,53 @@ packages: '@cfworker/json-schema': optional: true + '@smithy/core@3.33.3': + resolution: {integrity: sha512-CsOeKq/9kA3y6VJHt+/+VTCtBaxJ4OTFpgrjIUhPpDIKxBci1k2bJaQASF2h/ELWrulGp+t97DZ0mevfAD8idg==} + engines: {node: '>=18.0.0'} + + '@smithy/credential-provider-imds@4.5.2': + resolution: {integrity: sha512-A9uSdn72ozbRUSit0eib0TW7nXuNPlaeM0zcGkJ+nE6tFcSDbnmtwoxbTCFBukVQcszDAyvsd7+rTduPTXpygg==} + engines: {node: '>=18.0.0'} + + '@smithy/fetch-http-handler@5.8.0': + resolution: {integrity: sha512-ycSJu3tFAQ4v04CBB0agqFMVsSQ1iG3yw+SpgxRqKfaURpQD4CZ8Wn0zPMmSnOuTpTh65Vz+EA0rMrw089wvkA==} + engines: {node: '>=18.0.0'} + + '@smithy/hash-node@4.5.2': + resolution: {integrity: sha512-OcD8fGClTkP0BWHVEAgUp1RZyCw8cKfqTPQ+DgrSF5jvR8zKkw2Aud79L4G/1Fu3QKLcsHExxRIPQCcKx7+xkg==} + engines: {node: '>=18.0.0'} + + '@smithy/node-http-handler@4.12.1': + resolution: {integrity: sha512-ThMkboGeONWXAelq9FvGsuJC4rOi+qyC4/zhUF58xYpxUg5sQKx2VXZYJmtNjr4dSuBJ1HeJXETQILCz3wOHvw==} + engines: {node: '>=18.0.0'} + + '@smithy/protocol-http@5.6.2': + resolution: {integrity: sha512-Asd04MaxODN6FNY8EPTeCAM4kPNi3jDUAjZU0Y4F9rHvpLUrrUo7KLcxFgSthywFr6dZfIyDLIJda6jxmVTk5w==} + engines: {node: '>=18.0.0'} + + '@smithy/signature-v4@5.7.3': + resolution: {integrity: sha512-7ImGm+FkHRLcBaRttIAMZ6bzJZWb2cJGoYjq46F2UjycujWzrL9GEN9h4w7eQyXJYnltrUhxbbieBAIRrdqpow==} + engines: {node: '>=18.0.0'} + + '@smithy/types@4.18.0': + resolution: {integrity: sha512-CgB6HHWer/vrKps24ulRIbpcpb7K4xAU7SkZ7YHzBPlwHsvsrCJFEXK421s+cJzX+ZrqtA/TuU5w1HzI7k9N8A==} + engines: {node: '>=18.0.0'} + '@types/node@22.19.19': resolution: {integrity: sha512-dyh/xO2Fh5bYrfWaaqGrRQQGkNdmYw6AmaAUvYeUMNTWQtvb796ikLdmTchRmOlOiIJ1TDXfWgVx1QkUlQ6Hew==} + '@typespec/ts-http-runtime@0.3.9': + resolution: {integrity: sha512-edSdeAqkdxBVzA1yL1LrLCml1YjyCVvPMtMqJpbF+6K609tHe8V6sQUzFQSGcYNhcuhOceZtjvN32+mpIth30A==} + engines: {node: '>=22.0.0'} + accepts@2.0.0: resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==} engines: {node: '>= 0.6'} + agent-base@7.1.4: + resolution: {integrity: sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==} + engines: {node: '>= 14'} + ajv-formats@3.0.1: resolution: {integrity: sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==} peerDependencies: @@ -471,10 +641,26 @@ packages: ajv@8.20.0: resolution: {integrity: sha512-Thbli+OlOj+iMPYFBVBfJ3OmCAnaSyNn4M1vz9T6Gka5Jt9ba/HIR56joy65tY6kx/FCF5VXNB819Y7/GUrBGA==} + base64-js@1.5.1: + resolution: {integrity: sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==} + + bignumber.js@9.3.1: + resolution: {integrity: sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ==} + body-parser@2.3.0: resolution: {integrity: sha512-2cGmJupaNgg+QUwVLAucDuWuoMZ6EX9iHDRswZ5lsNYEmwPaRknMPCLZz07yTzVq/83p4o/wzbDZbBrTvGGTIw==} engines: {node: '>=18'} + bowser@2.14.1: + resolution: {integrity: sha512-tzPjzCxygAKWFOJP011oxFHs57HzIhOEracIgAePE4pqB3LikALKnSzUyU4MGs9/iCEUuHlAJTjTc5M+u7YEGg==} + + buffer-equal-constant-time@1.0.1: + resolution: {integrity: sha512-zRpUiDwd/xk6ADqPMATG8vc9VPrkck7T07OIx0gnjmJAnHnTVXNQG3vfvWNuiZIkwu9KrKdA1iJKfsfTVxE6NA==} + + bundle-name@4.1.0: + resolution: {integrity: sha512-tjwM5exMg6BGRI+kNmTntNsvdZS1X8BFYS6tnJ2hdH0kVxM6/eVZ2xy+FqStSWvYmtfFMDLIxurorHwDKfDz5Q==} + engines: {node: '>=18'} + bytes@3.1.2: resolution: {integrity: sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==} engines: {node: '>= 0.8'} @@ -515,6 +701,10 @@ packages: resolution: {integrity: sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==} engines: {node: '>= 8'} + data-uri-to-buffer@4.0.1: + resolution: {integrity: sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==} + engines: {node: '>= 12'} + debug@4.4.3: resolution: {integrity: sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==} engines: {node: '>=6.0'} @@ -524,6 +714,18 @@ packages: supports-color: optional: true + default-browser-id@5.0.1: + resolution: {integrity: sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q==} + engines: {node: '>=18'} + + default-browser@5.5.1: + resolution: {integrity: sha512-m1pAzaJgZ/gssEqlOhJkPJp8Xly7QyW6xcrkUa2KKcDeDSEMP7X8xipU3snUcfisTQx0w1AGae+9UtJSfVnXGw==} + engines: {node: '>=18'} + + define-lazy-prop@3.0.0: + resolution: {integrity: sha512-N+MeXYoqr3pOgn8xfyRPREN7gHakLYjhsHhWGT3fWAiL4IkAt0iDw14QiiEm2bE30c5XX5q0FtAA3CK5f9/BUg==} + engines: {node: '>=12'} + depd@2.0.0: resolution: {integrity: sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==} engines: {node: '>= 0.8'} @@ -532,6 +734,9 @@ packages: resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==} engines: {node: '>= 0.4'} + ecdsa-sig-formatter@1.0.11: + resolution: {integrity: sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==} + ee-first@1.1.1: resolution: {integrity: sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow==} @@ -581,16 +786,27 @@ packages: resolution: {integrity: sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw==} engines: {node: '>= 18'} + extend@3.0.2: + resolution: {integrity: sha512-fjquC59cD7CyW6urNXK0FBufkZcoiGG80wTuPujX590cB5Ttln20E2UB4S/WARVqhXffZl2LNgS+gQdPIIim/g==} + fast-deep-equal@3.1.3: resolution: {integrity: sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==} fast-uri@3.1.6: resolution: {integrity: sha512-7Ical1vFEMr0onbVzEDIreM22I4khW+fzyQPwvAFWBp1iwdshSZRsL4jjRvPG9JP1uiqMHRto+YU6R2/CzDz5Q==} + fetch-blob@3.2.0: + resolution: {integrity: sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==} + engines: {node: ^12.20 || >= 14.13} + finalhandler@2.1.1: resolution: {integrity: sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA==} engines: {node: '>= 18.0.0'} + formdata-polyfill@4.0.10: + resolution: {integrity: sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==} + engines: {node: '>=12.20.0'} + forwarded@0.2.0: resolution: {integrity: sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow==} engines: {node: '>= 0.6'} @@ -602,6 +818,14 @@ packages: function-bind@1.1.2: resolution: {integrity: sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==} + gaxios@7.3.1: + resolution: {integrity: sha512-kB3rzJV7d9juLZh8/56QTXCwQfxyhdOMdyYk1HdQKFtF8TJTDTZQJtixWIwXdE9Jji91mC41DUNpjleo4L4eAQ==} + engines: {node: '>=18'} + + gcp-metadata@9.0.3: + resolution: {integrity: sha512-2YYnIlHaKBGT2IPg3G2M57hia9Galz15zsEOvw9T3oRf0lSn6KN6VcHQLqby7x8ksYKnjXvp3rp2KJyLCN6zfQ==} + engines: {node: '>=22'} + get-intrinsic@1.3.0: resolution: {integrity: sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==} engines: {node: '>= 0.4'} @@ -610,6 +834,14 @@ packages: resolution: {integrity: sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==} engines: {node: '>= 0.4'} + google-auth-library@11.0.2: + resolution: {integrity: sha512-vzpgPutxrghPsnjrjpzLX2bdv8IOL719Rh0oEjGnQu8YCIbnbMuTTQ5zU9LcKvLdOPgCxBwppbvnhgW90Qna5Q==} + engines: {node: '>=22'} + + google-logging-utils@2.0.1: + resolution: {integrity: sha512-HMhaQghlOTvbcb3c4T5jmmOMtG3JUF1iOQMezaJXL86CDS+Tm2vHd0IeLFRAx3+ewd+bo9E1HFHoy17X5aJa9A==} + engines: {node: '>=22'} + gopd@1.2.0: resolution: {integrity: sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==} engines: {node: '>= 0.4'} @@ -638,6 +870,14 @@ packages: resolution: {integrity: sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==} engines: {node: '>= 0.8'} + http-proxy-agent@7.0.2: + resolution: {integrity: sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig==} + engines: {node: '>= 14'} + + https-proxy-agent@7.0.6: + resolution: {integrity: sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==} + engines: {node: '>= 14'} + iconv-lite@0.7.3: resolution: {integrity: sha512-IKXpvIzjnC9XTAUbVBcMfGS0EPaIXtW6v+zr+RRp+hqULEpo0owZax6wyRwPOJbWbzjYspQwusTsfVr0ifh4uQ==} engines: {node: '>=0.10.0'} @@ -653,21 +893,69 @@ packages: resolution: {integrity: sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g==} engines: {node: '>= 0.10'} + is-docker@3.0.0: + resolution: {integrity: sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + hasBin: true + + is-inside-container@1.0.0: + resolution: {integrity: sha512-KIYLCCJghfHZxqjYBE7rEy0OBuTd5xCHS7tHVgvCLkx7StIoaxwNW3hCALgEUjFfeRk+MG/Qxmp/vtETEF3tRA==} + engines: {node: '>=14.16'} + hasBin: true + is-promise@4.0.0: resolution: {integrity: sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ==} + is-wsl@3.1.1: + resolution: {integrity: sha512-e6rvdUCiQCAuumZslxRJWR/Doq4VpPR82kqclvcS0efgt430SlGIk05vdCN58+VrzgtIcfNODjozVielycD4Sw==} + engines: {node: '>=16'} + isexe@2.0.0: resolution: {integrity: sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw==} jose@6.2.10: resolution: {integrity: sha512-iiW7J9qRFlGxvCOIBDBDxFePQSn7ZMAnrYGhrrOo6siO/MIqwfyilLR27pkfDgUk+raLuzADS8A3S/KLBisc0g==} + json-bigint@1.0.0: + resolution: {integrity: sha512-SiPv/8VpZuWbvLSMtTDU8hEfrZWg/mH/nV/b4o0CYbSxu1UIQPLdwKOCIyLQX+VIPO5vrLX3i8qtqFyhdPSUSQ==} + json-schema-traverse@1.0.0: resolution: {integrity: sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug==} json-schema-typed@8.0.2: resolution: {integrity: sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA==} + jsonwebtoken@9.0.3: + resolution: {integrity: sha512-MT/xP0CrubFRNLNKvxJ2BYfy53Zkm++5bX9dtuPbqAeQpTVe0MQTFhao8+Cp//EmJp244xt6Drw/GVEGCUj40g==} + engines: {node: '>=12', npm: '>=6'} + + jwa@2.0.1: + resolution: {integrity: sha512-hRF04fqJIP8Abbkq5NKGN0Bbr3JxlQ+qhZufXVr0DvujKy93ZCbXZMHDL4EOtodSbCWxOqR8MS1tXA5hwqCXDg==} + + jws@4.0.1: + resolution: {integrity: sha512-EKI/M/yqPncGUUh44xz0PxSidXFr/+r0pA70+gIYhjv+et7yxM+s29Y+VGDkovRofQem0fs7Uvf4+YmAdyRduA==} + + lodash.includes@4.3.0: + resolution: {integrity: sha512-W3Bx6mdkRTGtlJISOvVD/lbqjTlPPUDTMnlXZFnVwi9NKJ6tiAk6LVdlhZMm17VZisqhKcgzpO5Wz91PCt5b0w==} + + lodash.isboolean@3.0.3: + resolution: {integrity: sha512-Bz5mupy2SVbPHURB98VAcw+aHh4vRV5IPNhILUCsOzRmsTmSQ17jIuqopAentWoehktxGd9e/hbIXq980/1QJg==} + + lodash.isinteger@4.0.4: + resolution: {integrity: sha512-DBwtEWN2caHQ9/imiNeEA5ys1JoRtRfY3d7V9wkqtbycnAmTvRRmbHKDV4a0EYc678/dia0jrte4tjYwVBaZUA==} + + lodash.isnumber@3.0.3: + resolution: {integrity: sha512-QYqzpfwO3/CWf3XP+Z+tkQsfaLL/EnUlXWVkIk5FUPc4sBdTehEqZONuyRt2P67PXAk+NXmTBcc97zw9t1FQrw==} + + lodash.isplainobject@4.0.6: + resolution: {integrity: sha512-oSXzaWypCMHkPC3NvBEaPHf0KsA5mvPrOPgQWDsbg8n7orZ290M0BmC/jgRZ4vcJ6DTAhjrsSYgdsW/F+MFOBA==} + + lodash.isstring@4.0.1: + resolution: {integrity: sha512-0wJxfxH1wgO3GrbuP+dTTk7op+6L41QCXbGINEmD+ny/G/eCqGzxyCsh7159S+mgDDcoarnBw6PC1PS5+wUGgw==} + + lodash.once@4.1.1: + resolution: {integrity: sha512-Sb487aTOCr9drQVL8pIxOzVhafOjZN9UU54hiN8PU3uAiSV7lx1yYNpbNmex2PK6dSJoNTSJUUswT651yww3Mg==} + marked@18.0.11: resolution: {integrity: sha512-HnslJfsZkRPBDJRHvVtAaWlZHEpSu7u8LgQuJCELjRKuWR+hpq4A7sLq3p8HaI9ypVoXDXxV34CsQJEe1+J5Aw==} engines: {node: '>= 20'} @@ -700,6 +988,15 @@ packages: resolution: {integrity: sha512-NMPBRMJgiQHjbd8phG3Vebdx4kZ1H121rbl5IkMqeOsahptB9BKo/d7oJ3zTXqTgagn2bWlNSXkh0QUGM31RYg==} engines: {node: '>=18'} + node-domexception@1.0.0: + resolution: {integrity: sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==} + engines: {node: '>=10.5.0'} + deprecated: Use your platform's native DOMException instead + + node-fetch@3.3.2: + resolution: {integrity: sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==} + engines: {node: ^12.20.0 || ^14.13.1 || >=16.0.0} + object-assign@4.1.1: resolution: {integrity: sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg==} engines: {node: '>=0.10.0'} @@ -715,6 +1012,10 @@ packages: once@1.4.0: resolution: {integrity: sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w==} + open@10.2.0: + resolution: {integrity: sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA==} + engines: {node: '>=18'} + parseurl@1.3.3: resolution: {integrity: sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ==} engines: {node: '>= 0.8'} @@ -754,9 +1055,21 @@ packages: resolution: {integrity: sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ==} engines: {node: '>= 18'} + run-applescript@7.1.0: + resolution: {integrity: sha512-DPe5pVFaAsinSaV6QjQ6gdiedWDcRCbUuiQfQa2wmWV7+xC9bGulGI8+TdRmoFkAPaBXk8CrAbnlY2ISniJ47Q==} + engines: {node: '>=18'} + + safe-buffer@5.2.1: + resolution: {integrity: sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ==} + safer-buffer@2.1.2: resolution: {integrity: sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==} + semver@7.8.5: + resolution: {integrity: sha512-Y7/KDsb8LjooZpwaqGyulO6DQlksgCncchHGk+sZIY4SBvUocMBEFH5Ur1fI4dV+Jvl0w6cjvucaIi40puRioA==} + engines: {node: '>=10'} + hasBin: true + send@1.2.1: resolution: {integrity: sha512-1gnZf7DFcoIcajTjTwjwuDjzuz4PPcY2StKPlsGAQ1+YH20IRVrBaXSWmdjowTJ6u8Rc01PoYOGHXfP1mYcZNQ==} engines: {node: '>= 18'} @@ -800,6 +1113,9 @@ packages: resolution: {integrity: sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==} engines: {node: '>=0.6'} + tslib@2.8.1: + resolution: {integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==} + type-is@2.1.0: resolution: {integrity: sha512-faYHw0anBbc/kWF3zFTEnxSFOAGUX9GFbOBthvDdLsIlEoWOFOtS0zgCiQYwIskL9iGXZL3kAXD8OoZ4GmMATA==} engines: {node: '>= 18'} @@ -824,6 +1140,10 @@ packages: resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} engines: {node: '>= 0.8'} + web-streams-polyfill@3.3.3: + resolution: {integrity: sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==} + engines: {node: '>= 8'} + which@2.0.2: resolution: {integrity: sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA==} engines: {node: '>= 8'} @@ -832,6 +1152,10 @@ packages: wrappy@1.0.2: resolution: {integrity: sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==} + wsl-utils@0.1.0: + resolution: {integrity: sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw==} + engines: {node: '>=18'} + yaml@2.8.3: resolution: {integrity: sha512-AvbaCLOO2Otw/lW5bmh9d/WEdcDFdQp2Z2ZUH3pX9U2ihyUY0nvLv7J6TrWowklRGPYbB/IuIMfYgxaCPg5Bpg==} engines: {node: '>= 14.6'} @@ -847,6 +1171,227 @@ packages: snapshots: + '@aws-sdk/core@3.977.9': + dependencies: + '@aws-sdk/types': 3.974.5 + '@aws-sdk/xml-builder': 3.972.40 + '@aws/lambda-invoke-store': 0.3.0 + '@smithy/core': 3.33.3 + '@smithy/signature-v4': 5.7.3 + '@smithy/types': 4.18.0 + bowser: 2.14.1 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-env@3.972.70': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-http@3.972.72': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/fetch-http-handler': 5.8.0 + '@smithy/node-http-handler': 4.12.1 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-ini@3.973.15': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/credential-provider-env': 3.972.70 + '@aws-sdk/credential-provider-http': 3.972.72 + '@aws-sdk/credential-provider-login': 3.972.77 + '@aws-sdk/credential-provider-process': 3.972.70 + '@aws-sdk/credential-provider-sso': 3.973.14 + '@aws-sdk/credential-provider-web-identity': 3.972.76 + '@aws-sdk/nested-clients': 3.997.44 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/credential-provider-imds': 4.5.2 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-login@3.972.77': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/nested-clients': 3.997.44 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-node@3.972.82': + dependencies: + '@aws-sdk/credential-provider-env': 3.972.70 + '@aws-sdk/credential-provider-http': 3.972.72 + '@aws-sdk/credential-provider-ini': 3.973.15 + '@aws-sdk/credential-provider-process': 3.972.70 + '@aws-sdk/credential-provider-sso': 3.973.14 + '@aws-sdk/credential-provider-web-identity': 3.972.76 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/credential-provider-imds': 4.5.2 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-process@3.972.70': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-sso@3.973.14': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/nested-clients': 3.997.44 + '@aws-sdk/token-providers': 3.1116.0 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/credential-provider-web-identity@3.972.76': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/nested-clients': 3.997.44 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/nested-clients@3.997.44': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/signature-v4-multi-region': 3.996.46 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/fetch-http-handler': 5.8.0 + '@smithy/node-http-handler': 4.12.1 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/signature-v4-multi-region@3.996.46': + dependencies: + '@aws-sdk/types': 3.974.5 + '@smithy/signature-v4': 5.7.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/token-providers@3.1116.0': + dependencies: + '@aws-sdk/core': 3.977.9 + '@aws-sdk/nested-clients': 3.997.44 + '@aws-sdk/types': 3.974.5 + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/types@3.974.5': + dependencies: + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws-sdk/xml-builder@3.972.40': + dependencies: + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@aws/lambda-invoke-store@0.3.0': {} + + '@azure/abort-controller@2.2.0': + dependencies: + tslib: 2.8.1 + + '@azure/core-auth@1.11.0': + dependencies: + '@azure/abort-controller': 2.2.0 + '@azure/core-util': 1.14.0 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/core-client@1.11.1': + dependencies: + '@azure/abort-controller': 2.2.0 + '@azure/core-auth': 1.11.0 + '@azure/core-rest-pipeline': 1.25.0 + '@azure/core-tracing': 1.4.0 + '@azure/core-util': 1.14.0 + '@azure/logger': 1.4.0 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/core-process@1.0.0': {} + + '@azure/core-rest-pipeline@1.25.0': + dependencies: + '@azure/abort-controller': 2.2.0 + '@azure/core-auth': 1.11.0 + '@azure/core-tracing': 1.4.0 + '@azure/core-util': 1.14.0 + '@azure/logger': 1.4.0 + '@typespec/ts-http-runtime': 0.3.9 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/core-tracing@1.4.0': + dependencies: + tslib: 2.8.1 + + '@azure/core-util@1.14.0': + dependencies: + '@azure/abort-controller': 2.2.0 + '@typespec/ts-http-runtime': 0.3.9 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/identity@4.13.2': + dependencies: + '@azure/abort-controller': 2.2.0 + '@azure/core-auth': 1.11.0 + '@azure/core-client': 1.11.1 + '@azure/core-process': 1.0.0 + '@azure/core-rest-pipeline': 1.25.0 + '@azure/core-tracing': 1.4.0 + '@azure/core-util': 1.14.0 + '@azure/logger': 1.4.0 + '@azure/msal-browser': 5.21.0 + '@azure/msal-node': 5.6.0 + open: 10.2.0 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/logger@1.4.0': + dependencies: + '@typespec/ts-http-runtime': 0.3.9 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + + '@azure/msal-browser@5.21.0': + dependencies: + '@azure/msal-common': 16.14.0 + + '@azure/msal-common@16.13.0': {} + + '@azure/msal-common@16.14.0': {} + + '@azure/msal-node@5.6.0': + dependencies: + '@azure/msal-common': 16.13.0 + jsonwebtoken: 9.0.3 + '@biomejs/biome@2.3.5': optionalDependencies: '@biomejs/cli-darwin-arm64': 2.3.5 @@ -997,15 +1542,68 @@ snapshots: transitivePeerDependencies: - supports-color + '@smithy/core@3.33.3': + dependencies: + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@smithy/credential-provider-imds@4.5.2': + dependencies: + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@smithy/fetch-http-handler@5.8.0': + dependencies: + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@smithy/hash-node@4.5.2': + dependencies: + '@smithy/core': 3.33.3 + tslib: 2.8.1 + + '@smithy/node-http-handler@4.12.1': + dependencies: + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@smithy/protocol-http@5.6.2': + dependencies: + '@smithy/core': 3.33.3 + tslib: 2.8.1 + + '@smithy/signature-v4@5.7.3': + dependencies: + '@smithy/core': 3.33.3 + '@smithy/types': 4.18.0 + tslib: 2.8.1 + + '@smithy/types@4.18.0': + dependencies: + tslib: 2.8.1 + '@types/node@22.19.19': dependencies: undici-types: 6.21.0 + '@typespec/ts-http-runtime@0.3.9': + dependencies: + http-proxy-agent: 7.0.2 + https-proxy-agent: 7.0.6 + tslib: 2.8.1 + transitivePeerDependencies: + - supports-color + accepts@2.0.0: dependencies: mime-types: 3.0.2 negotiator: 1.1.0 + agent-base@7.1.4: {} + ajv-formats@3.0.1(ajv@8.20.0): optionalDependencies: ajv: 8.20.0 @@ -1017,6 +1615,10 @@ snapshots: json-schema-traverse: 1.0.0 require-from-string: 2.0.2 + base64-js@1.5.1: {} + + bignumber.js@9.3.1: {} + body-parser@2.3.0: dependencies: bytes: 3.1.2 @@ -1031,6 +1633,14 @@ snapshots: transitivePeerDependencies: - supports-color + bowser@2.14.1: {} + + buffer-equal-constant-time@1.0.1: {} + + bundle-name@4.1.0: + dependencies: + run-applescript: 7.1.0 + bytes@3.1.2: {} call-bind-apply-helpers@1.0.2: @@ -1064,10 +1674,21 @@ snapshots: shebang-command: 2.0.0 which: 2.0.2 + data-uri-to-buffer@4.0.1: {} + debug@4.4.3: dependencies: ms: 2.1.3 + default-browser-id@5.0.1: {} + + default-browser@5.5.1: + dependencies: + bundle-name: 4.1.0 + default-browser-id: 5.0.1 + + define-lazy-prop@3.0.0: {} + depd@2.0.0: {} dunder-proto@1.0.1: @@ -1076,6 +1697,10 @@ snapshots: es-errors: 1.3.0 gopd: 1.2.0 + ecdsa-sig-formatter@1.0.11: + dependencies: + safe-buffer: 5.2.1 + ee-first@1.1.1: {} encodeurl@2.0.0: {} @@ -1168,10 +1793,17 @@ snapshots: transitivePeerDependencies: - supports-color + extend@3.0.2: {} + fast-deep-equal@3.1.3: {} fast-uri@3.1.6: {} + fetch-blob@3.2.0: + dependencies: + node-domexception: 1.0.0 + web-streams-polyfill: 3.3.3 + finalhandler@2.1.1: dependencies: debug: 4.4.3 @@ -1183,12 +1815,32 @@ snapshots: transitivePeerDependencies: - supports-color + formdata-polyfill@4.0.10: + dependencies: + fetch-blob: 3.2.0 + forwarded@0.2.0: {} fresh@2.0.0: {} function-bind@1.1.2: {} + gaxios@7.3.1: + dependencies: + extend: 3.0.2 + https-proxy-agent: 7.0.6 + node-fetch: 3.3.2 + transitivePeerDependencies: + - supports-color + + gcp-metadata@9.0.3: + dependencies: + gaxios: 7.3.1 + google-logging-utils: 2.0.1 + json-bigint: 1.0.0 + transitivePeerDependencies: + - supports-color + get-intrinsic@1.3.0: dependencies: call-bind-apply-helpers: 1.0.2 @@ -1207,6 +1859,19 @@ snapshots: dunder-proto: 1.0.1 es-object-atoms: 1.1.2 + google-auth-library@11.0.2: + dependencies: + base64-js: 1.5.1 + ecdsa-sig-formatter: 1.0.11 + gaxios: 7.3.1 + gcp-metadata: 9.0.3 + google-logging-utils: 2.0.1 + jws: 4.0.1 + transitivePeerDependencies: + - supports-color + + google-logging-utils@2.0.1: {} + gopd@1.2.0: {} grok-mermaid@0.2.2: {} @@ -1229,6 +1894,20 @@ snapshots: statuses: 2.0.2 toidentifier: 1.0.1 + http-proxy-agent@7.0.2: + dependencies: + agent-base: 7.1.4 + debug: 4.4.3 + transitivePeerDependencies: + - supports-color + + https-proxy-agent@7.0.6: + dependencies: + agent-base: 7.1.4 + debug: 4.4.3 + transitivePeerDependencies: + - supports-color + iconv-lite@0.7.3: dependencies: safer-buffer: 2.1.2 @@ -1239,16 +1918,68 @@ snapshots: ipaddr.js@1.9.1: {} + is-docker@3.0.0: {} + + is-inside-container@1.0.0: + dependencies: + is-docker: 3.0.0 + is-promise@4.0.0: {} + is-wsl@3.1.1: + dependencies: + is-inside-container: 1.0.0 + isexe@2.0.0: {} jose@6.2.10: {} + json-bigint@1.0.0: + dependencies: + bignumber.js: 9.3.1 + json-schema-traverse@1.0.0: {} json-schema-typed@8.0.2: {} + jsonwebtoken@9.0.3: + dependencies: + jws: 4.0.1 + lodash.includes: 4.3.0 + lodash.isboolean: 3.0.3 + lodash.isinteger: 4.0.4 + lodash.isnumber: 3.0.3 + lodash.isplainobject: 4.0.6 + lodash.isstring: 4.0.1 + lodash.once: 4.1.1 + ms: 2.1.3 + semver: 7.8.5 + + jwa@2.0.1: + dependencies: + buffer-equal-constant-time: 1.0.1 + ecdsa-sig-formatter: 1.0.11 + safe-buffer: 5.2.1 + + jws@4.0.1: + dependencies: + jwa: 2.0.1 + safe-buffer: 5.2.1 + + lodash.includes@4.3.0: {} + + lodash.isboolean@3.0.3: {} + + lodash.isinteger@4.0.4: {} + + lodash.isnumber@3.0.3: {} + + lodash.isplainobject@4.0.6: {} + + lodash.isstring@4.0.1: {} + + lodash.once@4.1.1: {} + marked@18.0.11: {} math-intrinsics@1.1.0: {} @@ -1269,6 +2000,14 @@ snapshots: dependencies: content-type: 2.1.0 + node-domexception@1.0.0: {} + + node-fetch@3.3.2: + dependencies: + data-uri-to-buffer: 4.0.1 + fetch-blob: 3.2.0 + formdata-polyfill: 4.0.10 + object-assign@4.1.1: {} object-inspect@1.13.4: {} @@ -1281,6 +2020,13 @@ snapshots: dependencies: wrappy: 1.0.2 + open@10.2.0: + dependencies: + default-browser: 5.5.1 + define-lazy-prop: 3.0.0 + is-inside-container: 1.0.0 + wsl-utils: 0.1.0 + parseurl@1.3.3: {} path-key@3.1.1: {} @@ -1320,8 +2066,14 @@ snapshots: transitivePeerDependencies: - supports-color + run-applescript@7.1.0: {} + + safe-buffer@5.2.1: {} + safer-buffer@2.1.2: {} + semver@7.8.5: {} + send@1.2.1: dependencies: debug: 4.4.3 @@ -1387,6 +2139,8 @@ snapshots: toidentifier@1.0.1: {} + tslib@2.8.1: {} + type-is@2.1.0: dependencies: content-type: 2.1.0 @@ -1403,12 +2157,18 @@ snapshots: vary@1.1.2: {} + web-streams-polyfill@3.3.3: {} + which@2.0.2: dependencies: isexe: 2.0.0 wrappy@1.0.2: {} + wsl-utils@0.1.0: + dependencies: + is-wsl: 3.1.1 + yaml@2.8.3: {} zod-to-json-schema@3.25.2(zod@4.5.4):