diff --git a/packages/ai-sdk-provider/README.md b/packages/ai-sdk-provider/README.md index f82924f9b6..86bd413fb8 100644 --- a/packages/ai-sdk-provider/README.md +++ b/packages/ai-sdk-provider/README.md @@ -2,9 +2,11 @@ [Vercel AI SDK](https://ai-sdk.dev) provider for the [QVAC](https://qvac.tether.io) local AI runtime. -QVAC is an open-source, cross-platform ecosystem for **local-first, peer-to-peer AI** — LLMs, embeddings, transcription, translation, speech, OCR, and image generation, all running on the user's own hardware. This package is a thin, branded wrapper around [`@ai-sdk/openai-compatible`](https://www.npmjs.com/package/@ai-sdk/openai-compatible) that points at a running `qvac serve openai` HTTP server and re-exports QVAC's model metadata so callers can introspect typed model constants without an HTTP round-trip. +QVAC is an open-source, cross-platform ecosystem for **local-first, peer-to-peer AI** — LLMs, embeddings, transcription, translation, speech, OCR, and image generation, all running on the user's own hardware. This package implements the native provider contracts for language, embedding, image, file-reference, transcription, and speech operations against a local `qvac serve openai` runtime. It also re-exports QVAC's model metadata so callers can introspect typed model constants without an HTTP round-trip. -> **Status — `0.2.0`.** Two modes: +The provider is OpenAI-compatible by construction: the OpenAI-compatible transport is its fallback provider, while the SDK's native custom-provider composition adds QVAC-specific files, transcription, and speech capabilities. Existing OpenAI-shaped chat, completion, embedding, and image requests keep the same wire protocol. + +> **Runtime modes:** > > - **External** (default): the package wraps a `qvac serve openai` HTTP endpoint that you run yourself. > - **Managed** (`mode: 'managed'`): the provider synthesizes an ephemeral config from a model list, then spawns (or reuses) a shared `qvac serve` on a free port and keeps it alive for as long as anything is using it, reaping it automatically once everyone is done. See [Managed mode](#managed-mode) below. Requires the optional [`@qvac/cli`](https://www.npmjs.com/package/@qvac/cli) peer dependency. @@ -20,7 +22,14 @@ bun add @qvac/ai-sdk-provider ai @ai-sdk/openai-compatible # or: npm install @qvac/ai-sdk-provider ai @ai-sdk/openai-compatible ``` -`ai` and `@ai-sdk/openai-compatible` are **peer dependencies** — install them alongside. +Runtime requirements: + +- **Node.js 22 or newer.** Node 20 is not supported. +- **AI SDK 7** (`ai@^7`) and **OpenAI-compatible provider 3** (`@ai-sdk/openai-compatible@^3`). +- **Provider-v4 contracts.** Custom middleware and direct model integrations must use the AI SDK v4 provider interfaces exposed by these versions. +- **ESM-only imports.** Use `import` / dynamic `import()`; CommonJS `require()` is not supported. + +`ai` and `@ai-sdk/openai-compatible` are **peer dependencies** — install those compatible major versions alongside the provider. --- @@ -77,8 +86,65 @@ qvac.chatModel('qwen3-600m') // explicit chat model qvac.completionModel('qwen3-600m') // legacy completion qvac.textEmbeddingModel('embed-gemma') // text embeddings qvac.imageModel('flux-schnell') // image generation +qvac.transcriptionModel('whisper') // speech to text +qvac.speechModel('tts') // text to speech +qvac.files() // local ephemeral file uploads ``` +### Local file references + +`uploadFile` stores bytes in the running QVAC process and returns a `qvac` provider reference. Passing that reference to a QVAC language model resolves the bytes through the same local runtime immediately before inference. The reference never becomes a cloud URL and expires with QVAC's ephemeral file store. + +```ts +import { readFileSync } from 'node:fs' +import { generateText, uploadFile } from 'ai' + +const { providerReference } = await uploadFile({ + api: qvac, + data: readFileSync('./sensor.png'), + mediaType: 'image/png', + filename: 'sensor.png' +}) + +const { text } = await generateText({ + model: qvac('vision'), + messages: [ + { + role: 'user', + content: [ + { type: 'text', text: 'Read the sensor display.' }, + { type: 'file', mediaType: 'image/png', data: providerReference } + ] + } + ] +}) +``` + +### Native speech and transcription + +Speech operations use the provider's native model contracts while remaining entirely inside the local QVAC serve: + +```ts +import { generateSpeech, transcribe } from 'ai' + +const speech = await generateSpeech({ + model: qvac.speechModel('tts'), + text: 'The sensor temperature is twenty four degrees.', + voice: 'alloy', + outputFormat: 'wav' +}) + +const transcript = await transcribe({ + model: qvac.transcriptionModel('stt'), + audio: speech.audio.uint8Array, + providerOptions: { + qvac: { prompt: 'Sensor terminology' } + } +}) +``` + +Per-request transcription `prompt` is supported. Language is selected when the QVAC model is loaded, and speech speed, instructions, and language are not consumed by the current local engines; the provider returns structured warnings when callers supply those options. + --- ## Managed mode @@ -187,7 +253,7 @@ Managed mode runs `qvac serve` as a **shared, self-cleaning daemon** so that ope - **Startup is gated on model preload.** `qvac serve` does not open its port until every preloaded model is ready, and a cold P2P download can take minutes — hence the generous default `serveStartTimeout`. Raise it for large models. - **External mode pays nothing.** The managed subsystem (and its `node:child_process` / `@qvac/cli` resolution) is dynamically imported only when `mode: 'managed'` is set. -- **Node 20+ and Bun.** The managed subsystem uses only portable `node:` APIs — no Bun-specific calls. +- **Node 22+ and Bun.** The managed subsystem uses only portable `node:` APIs — no Bun-specific calls. - **Typed errors.** Managed setup throws structured errors you can `instanceof`-check: `UnknownManagedModelError`, `DuplicateManagedModelError`, `MultipleDefaultManagedModelsError`, `CliNotFoundError`, `ServeStartTimeoutError`, `ServeSpawnFailedError`, `ServeExitedError`, and `PortAllocationFailedError` (all extending `QvacManagedModeError`, with a `.code` from `QvacManagedErrorCode`). They're exported from the package root. --- diff --git a/packages/ai-sdk-provider/package.json b/packages/ai-sdk-provider/package.json index f01d17f491..e7af85a6b6 100644 --- a/packages/ai-sdk-provider/package.json +++ b/packages/ai-sdk-provider/package.json @@ -27,7 +27,7 @@ "NOTICE" ], "engines": { - "node": ">=20.0.0" + "node": ">=22.0.0" }, "keywords": [ "ai", @@ -72,20 +72,24 @@ "test:integration": "tsx --test test/managed-integration.test.ts" }, "peerDependencies": { - "@ai-sdk/openai-compatible": "^2.0", + "@ai-sdk/openai-compatible": "^3.0", "@qvac/cli": "^0.6.0 || ^0.7.0 || ^0.8.0", - "ai": "^6.0" + "ai": "^7.0" }, "peerDependenciesMeta": { "@qvac/cli": { "optional": true } }, + "dependencies": { + "@ai-sdk/provider": "^4.0.0", + "@ai-sdk/provider-utils": "^5.0.0" + }, "devDependencies": { - "@ai-sdk/openai-compatible": "^2.0", + "@ai-sdk/openai-compatible": "^3.0", "@qvac/registry-client": "^0.5.0", "@types/node": "^24.2.1", - "ai": "^6.0", + "ai": "^7.0", "lunte": "^1.8.2", "prettier": "^3.9.4", "prettier-config-holepunch": "^2.0.0", diff --git a/packages/ai-sdk-provider/src/audio-models.ts b/packages/ai-sdk-provider/src/audio-models.ts new file mode 100644 index 0000000000..6c9bd117c1 --- /dev/null +++ b/packages/ai-sdk-provider/src/audio-models.ts @@ -0,0 +1,208 @@ +import type { SharedV4Warning, SpeechModelV4, TranscriptionModelV4 } from '@ai-sdk/provider' +import { + createBinaryResponseHandler, + createStatusCodeErrorResponseHandler, + postFormDataToApi, + postJsonToApi, + type ResponseHandler +} from '@ai-sdk/provider-utils' + +interface QvacAudioOptions { + readonly baseURL: string + readonly headers: Record + readonly fetch?: typeof fetch +} + +function endpoint(options: QvacAudioOptions, path: string): string { + return `${options.baseURL.replace(/\/+$/, '')}${path}` +} + +function requestHeaders( + configured: Record, + perCall: Record | undefined +): Record { + const headers = { ...configured } + for (const [name, value] of Object.entries(perCall ?? {})) { + if (value !== undefined) headers[name] = value + } + return headers +} + +function withoutContentType(headers: Record): Record { + return Object.fromEntries( + Object.entries(headers).filter(([name]) => name.toLowerCase() !== 'content-type') + ) +} + +function bytesFor(data: Uint8Array | string): Uint8Array { + return typeof data === 'string' ? Uint8Array.from(Buffer.from(data, 'base64')) : data +} + +interface JsonResponse { + readonly body: T + readonly headers: Record +} + +function jsonResponseHandler(): ResponseHandler> { + return async ({ response }) => { + const body = (await response.json()) as T + const headers = Object.fromEntries(response.headers.entries()) + return { + value: { body, headers }, + rawValue: body, + responseHeaders: headers + } + } +} + +const binaryResponseHandler = createBinaryResponseHandler() +const binaryWithHeadersHandler: ResponseHandler<{ + audio: Uint8Array + headers: Record +}> = async (options) => { + const result = await binaryResponseHandler(options) + return { + ...result, + value: { audio: result.value, headers: result.responseHeaders ?? {} } + } +} + +const statusCodeErrorResponseHandler = createStatusCodeErrorResponseHandler() + +export function createQvacTranscriptionModel( + modelId: string, + options: QvacAudioOptions +): TranscriptionModelV4 { + return { + specificationVersion: 'v4', + provider: 'qvac.transcription', + modelId, + async doGenerate({ audio, mediaType, providerOptions, abortSignal, headers }) { + const qvacOptions = providerOptions?.['qvac'] ?? {} + const form = new FormData() + const bytes = bytesFor(audio) + form.append( + 'file', + new Blob([bytes.slice().buffer as ArrayBuffer], { type: mediaType }), + 'audio-input' + ) + form.append('model', modelId) + form.append('response_format', 'json') + + const prompt = qvacOptions['prompt'] + const language = qvacOptions['language'] + const temperature = qvacOptions['temperature'] + if (typeof prompt === 'string') form.append('prompt', prompt) + if (typeof language === 'string') form.append('language', language) + if (typeof temperature === 'number') form.append('temperature', String(temperature)) + + const warnings: SharedV4Warning[] = [] + if (language !== undefined) { + warnings.push({ + type: 'unsupported', + feature: 'providerOptions.qvac.language', + details: 'QVAC configures transcription language when the model is loaded.' + }) + } + if (temperature !== undefined) { + warnings.push({ + type: 'unsupported', + feature: 'providerOptions.qvac.temperature', + details: 'The local transcription runtime does not use per-request temperature.' + }) + } + + const timestamp = new Date() + const transcriptionResponse = await postFormDataToApi({ + url: endpoint(options, '/audio/transcriptions'), + headers: withoutContentType(requestHeaders(options.headers, headers)), + formData: form, + failedResponseHandler: statusCodeErrorResponseHandler, + successfulResponseHandler: jsonResponseHandler<{ text?: unknown }>(), + ...(abortSignal !== undefined && { abortSignal }), + ...(options.fetch !== undefined && { fetch: options.fetch }) + }) + const { body, headers: responseHeaders } = transcriptionResponse.value + if (typeof body.text !== 'string') throw new Error('QVAC transcription returned no text') + + return { + text: body.text, + segments: [], + language: undefined, + durationInSeconds: undefined, + warnings, + response: { + timestamp, + modelId, + headers: responseHeaders, + body + }, + providerMetadata: { qvac: { local: true } } + } + } + } +} + +export function createQvacSpeechModel(modelId: string, options: QvacAudioOptions): SpeechModelV4 { + return { + specificationVersion: 'v4', + provider: 'qvac.speech', + modelId, + async doGenerate({ + text, + voice, + outputFormat, + instructions, + speed, + language, + abortSignal, + headers + }) { + const warnings: SharedV4Warning[] = [] + for (const [feature, value] of Object.entries({ instructions, speed, language })) { + if (value !== undefined) { + warnings.push({ + type: 'unsupported', + feature, + details: `QVAC local speech synthesis does not use per-request ${feature}.` + }) + } + } + + const body = { + model: modelId, + input: text, + ...(voice !== undefined && { voice }), + ...(outputFormat !== undefined && { response_format: outputFormat }) + } + const timestamp = new Date() + const speechResponse = await postJsonToApi({ + url: endpoint(options, '/audio/speech'), + headers: withoutContentType(requestHeaders(options.headers, headers)), + body, + failedResponseHandler: statusCodeErrorResponseHandler, + successfulResponseHandler: binaryWithHeadersHandler, + ...(abortSignal !== undefined && { abortSignal }), + ...(options.fetch !== undefined && { fetch: options.fetch }) + }) + const { audio, headers: responseHeaders } = speechResponse.value + + return { + audio, + warnings, + request: { body }, + response: { + timestamp, + modelId, + headers: responseHeaders + }, + providerMetadata: { + qvac: { + local: true, + mediaType: responseHeaders['content-type'] ?? 'application/octet-stream' + } + } + } + } + } +} diff --git a/packages/ai-sdk-provider/src/file-reference-model.ts b/packages/ai-sdk-provider/src/file-reference-model.ts new file mode 100644 index 0000000000..eaaa7f925e --- /dev/null +++ b/packages/ai-sdk-provider/src/file-reference-model.ts @@ -0,0 +1,75 @@ +import type { + LanguageModelV4, + LanguageModelV4CallOptions, + LanguageModelV4Prompt +} from '@ai-sdk/provider' +import { + createBinaryResponseHandler, + createStatusCodeErrorResponseHandler, + getFromApi, + resolveProviderReference +} from '@ai-sdk/provider-utils' + +import { mergeHeaders } from './headers.js' + +interface ResolveOptions { + readonly baseURL: string + readonly headers: Record + readonly fetch?: typeof fetch +} + +async function resolvePrompt( + prompt: LanguageModelV4Prompt, + options: ResolveOptions, + signal?: AbortSignal, + headers?: Record +): Promise { + const failedResponseHandler = createStatusCodeErrorResponseHandler() + const successfulResponseHandler = createBinaryResponseHandler() + return await Promise.all( + prompt.map(async (message) => { + if (message.role === 'system' || message.role === 'tool') return message + const content = await Promise.all( + message.content.map(async (part) => { + if (part.type !== 'file' || part.data.type !== 'reference') return part + const id = resolveProviderReference({ reference: part.data.reference, provider: 'qvac' }) + const response = await getFromApi({ + url: `${options.baseURL.replace(/\/+$/, '')}/files/${encodeURIComponent(id)}/content`, + headers: mergeHeaders(options.headers, headers), + failedResponseHandler, + successfulResponseHandler, + ...(signal !== undefined && { abortSignal: signal }), + ...(options.fetch !== undefined && { fetch: options.fetch }), + validateUrl: false + }) + return { + ...part, + data: { type: 'data' as const, data: response.value } + } + }) + ) + return { ...message, content } as typeof message + }) + ) +} + +/** Resolve QVAC provider references locally before the OpenAI-compatible wire conversion. */ +export function withQvacFileReferences( + model: LanguageModelV4, + options: ResolveOptions +): LanguageModelV4 { + const prepare = async ( + call: LanguageModelV4CallOptions + ): Promise => ({ + ...call, + prompt: await resolvePrompt(call.prompt, options, call.abortSignal, call.headers) + }) + return { + specificationVersion: 'v4', + provider: model.provider, + modelId: model.modelId, + supportedUrls: model.supportedUrls, + doGenerate: async (call) => model.doGenerate(await prepare(call)), + doStream: async (call) => model.doStream(await prepare(call)) + } +} diff --git a/packages/ai-sdk-provider/src/files.ts b/packages/ai-sdk-provider/src/files.ts new file mode 100644 index 0000000000..f0825fd794 --- /dev/null +++ b/packages/ai-sdk-provider/src/files.ts @@ -0,0 +1,67 @@ +import type { FilesV4, FilesV4UploadFileCallOptions } from '@ai-sdk/provider' +import { + createStatusCodeErrorResponseHandler, + postFormDataToApi, + type ResponseHandler +} from '@ai-sdk/provider-utils' + +export interface QvacFilesOptions { + readonly baseURL: string + readonly headers: Record + readonly fetch?: typeof fetch +} + +function multipartHeaders(headers: Record): Record { + return Object.fromEntries( + Object.entries(headers).filter(([name]) => name.toLowerCase() !== 'content-type') + ) +} + +function bytesFor(data: FilesV4UploadFileCallOptions['data']): Uint8Array { + if (data.type === 'text') return new TextEncoder().encode(data.text) + if (typeof data.data === 'string') return Uint8Array.from(Buffer.from(data.data, 'base64')) + return data.data +} + +/** AI SDK v4 files interface backed by QVAC serve's local ephemeral file store. */ +export function createQvacFiles(options: QvacFilesOptions): FilesV4 { + const failedResponseHandler = createStatusCodeErrorResponseHandler() + const successfulResponseHandler: ResponseHandler<{ id?: string; filename?: string }> = async ({ + response + }) => { + const result = (await response.json()) as { id?: string; filename?: string } + const responseHeaders = Object.fromEntries(response.headers.entries()) + return { value: result, rawValue: result, responseHeaders } + } + return { + specificationVersion: 'v4', + provider: 'qvac', + async uploadFile({ data, mediaType, filename, providerOptions }) { + const form = new FormData() + const bytes = bytesFor(data) + const body = bytes.slice().buffer as ArrayBuffer + form.append('file', new Blob([body], { type: mediaType }), filename ?? 'upload.bin') + const purpose = providerOptions?.['qvac']?.['purpose'] + if (typeof purpose === 'string' && purpose.length > 0) form.append('purpose', purpose) + + const response = await postFormDataToApi({ + url: `${options.baseURL.replace(/\/+$/, '')}/files`, + headers: multipartHeaders(options.headers), + formData: form, + failedResponseHandler, + successfulResponseHandler, + ...(options.fetch !== undefined && { fetch: options.fetch }) + }) + const result = response.value + if (!result.id) throw new Error('QVAC file upload returned no file id') + const resolvedFilename = result.filename ?? filename + return { + providerReference: { qvac: result.id }, + mediaType, + ...(resolvedFilename !== undefined && { filename: resolvedFilename }), + providerMetadata: { qvac: { ephemeral: true } }, + warnings: [] + } + } + } +} diff --git a/packages/ai-sdk-provider/src/headers.ts b/packages/ai-sdk-provider/src/headers.ts new file mode 100644 index 0000000000..db475571f9 --- /dev/null +++ b/packages/ai-sdk-provider/src/headers.ts @@ -0,0 +1,13 @@ +type HeaderSource = Readonly> | undefined + +/** Merge header records case-insensitively, with later sources taking precedence. */ +export function mergeHeaders(...sources: HeaderSource[]): Record { + const merged = new Headers() + for (const source of sources) { + if (source === undefined) continue + for (const [name, value] of Object.entries(source)) { + if (value !== undefined) merged.set(name, value) + } + } + return Object.fromEntries(merged.entries()) +} diff --git a/packages/ai-sdk-provider/src/managed/index.ts b/packages/ai-sdk-provider/src/managed/index.ts index bbeac1cf70..20406f4162 100644 --- a/packages/ai-sdk-provider/src/managed/index.ts +++ b/packages/ai-sdk-provider/src/managed/index.ts @@ -3,8 +3,6 @@ import { randomBytes } from 'node:crypto' import { mkdir, open, readFile, rm, stat } from 'node:fs/promises' import { join } from 'node:path' -import { createOpenAICompatible } from '@ai-sdk/openai-compatible' - import { DEFAULT_API_KEY, DEFAULT_HEADERS, @@ -15,6 +13,7 @@ import { SERVE_HEALTH_POLL_INTERVAL_MS, SPAWN_LOCK_STALE_MS } from '../defaults.js' +import { createExternalQvac } from '../provider.js' import type { ManagedQvacProvider, QvacManagedOptions } from '../types.js' import { synthesizeServeConfig, writeEphemeralConfig } from './config-synthesizer.js' import { ServeSpawnFailedError, ServeStartTimeoutError } from './errors.js' @@ -323,8 +322,7 @@ export async function startManagedQvac(options: QvacManagedOptions): Promise[0] = { name: 'qvac', - baseURL: options.baseURL ?? DEFAULT_BASE_URL, - apiKey: options.apiKey ?? DEFAULT_API_KEY, + baseURL, headers } if (options.fetch !== undefined) init.fetch = options.fetch - return createOpenAICompatible(init) as QvacProvider + const compatible = createOpenAICompatible(init) + const localOptions = { + baseURL, + headers, + ...(options.fetch !== undefined && { fetch: options.fetch }) + } + const wrap = (model: ReturnType) => + withQvacFileReferences(model, localOptions) + + // QVAC's HTTP transport remains OpenAI-compatible. `customProvider` composes + // the additional native capability contracts over that fallback instead of + // replacing or forking the compatible language/embedding/image behavior. + const compatibleFallback: ProviderV4 = { + specificationVersion: 'v4', + languageModel: (modelId) => wrap(compatible.languageModel(modelId)), + embeddingModel: (modelId) => compatible.embeddingModel(modelId), + imageModel: (modelId) => compatible.imageModel(modelId), + transcriptionModel: (modelId) => createQvacTranscriptionModel(modelId, localOptions), + speechModel: (modelId) => createQvacSpeechModel(modelId, localOptions) + } + const composed = customProvider({ + files: createQvacFiles(localOptions), + fallbackProvider: compatibleFallback + }) + + const provider = ((modelId: string) => composed.languageModel(modelId)) as QvacProvider + Object.assign(provider, compatible) + Object.defineProperty(provider, 'specificationVersion', { + value: composed.specificationVersion, + enumerable: true + }) + provider.languageModel = (modelId: string) => composed.languageModel(modelId) + provider.chatModel = (modelId: string) => composed.languageModel(modelId) + provider.embeddingModel = (modelId: string) => composed.embeddingModel(modelId) + provider.textEmbeddingModel = (modelId: string) => composed.embeddingModel(modelId) + provider.imageModel = (modelId: string) => composed.imageModel(modelId) + provider.transcriptionModel = (modelId: string) => composed.transcriptionModel(modelId) + provider.speechModel = (modelId: string) => composed.speechModel(modelId) + provider.files = () => composed.files() + return provider } export function createQvac(options?: QvacExternalOptions): QvacProvider diff --git a/packages/ai-sdk-provider/src/types.ts b/packages/ai-sdk-provider/src/types.ts index b0bd0080ca..8046b1390b 100644 --- a/packages/ai-sdk-provider/src/types.ts +++ b/packages/ai-sdk-provider/src/types.ts @@ -1,4 +1,5 @@ import type { OpenAICompatibleProvider } from '@ai-sdk/openai-compatible' +import type { FilesV4, SpeechModelV4, TranscriptionModelV4 } from '@ai-sdk/provider' // Options shared by every mode. `mode` is the discriminant; it defaults to // `'external'` so existing v1 callers (which never passed `mode`) keep the @@ -91,6 +92,9 @@ export type QvacOptions = QvacExternalOptions | QvacManagedOptions // TypeScript without paying runtime cost. export type QvacProvider = OpenAICompatibleProvider & { readonly _brand: 'qvac' + files(): FilesV4 + transcriptionModel(modelId: string): TranscriptionModelV4 + speechModel(modelId: string): SpeechModelV4 } // Provider returned by managed mode. Carries the live serve coordinates plus a diff --git a/packages/ai-sdk-provider/test/managed-provider.test.ts b/packages/ai-sdk-provider/test/managed-provider.test.ts index 79e5cb221f..102ee59967 100644 --- a/packages/ai-sdk-provider/test/managed-provider.test.ts +++ b/packages/ai-sdk-provider/test/managed-provider.test.ts @@ -37,6 +37,9 @@ test( assert.equal(typeof provider, 'function') assert.equal(typeof provider.chatModel, 'function') + assert.equal(typeof provider.files, 'function') + assert.equal(typeof provider.transcriptionModel, 'function') + assert.equal(typeof provider.speechModel, 'function') assert.equal(typeof provider.close, 'function') assert.equal(typeof provider[Symbol.asyncDispose], 'function') assert.ok(provider.port > 0) diff --git a/packages/ai-sdk-provider/test/provider.test.ts b/packages/ai-sdk-provider/test/provider.test.ts index 29b1262c6d..d23bfd5041 100644 --- a/packages/ai-sdk-provider/test/provider.test.ts +++ b/packages/ai-sdk-provider/test/provider.test.ts @@ -1,17 +1,300 @@ import assert from 'node:assert/strict' import test from 'node:test' +import { APICallError } from '@ai-sdk/provider' + import { DEFAULT_API_KEY, DEFAULT_BASE_URL } from '../src/defaults.js' import { createQvac, qvac } from '../src/provider.js' +async function captureApiCallError(promise: PromiseLike): Promise { + try { + await promise + assert.fail('expected APICallError') + } catch (error) { + assert.ok(APICallError.isInstance(error), `expected APICallError, got ${String(error)}`) + return error + } +} + test('createQvac returns a provider object with the AI SDK provider surface', () => { const provider = createQvac({ baseURL: 'http://127.0.0.1:55555/v1' }) assert.equal(typeof provider, 'function', 'provider should be callable as `provider(modelId)`') + assert.equal(provider.specificationVersion, 'v4') assert.equal(typeof provider.chatModel, 'function') assert.equal(typeof provider.completionModel, 'function') assert.equal(typeof provider.textEmbeddingModel, 'function') assert.equal(typeof provider.imageModel, 'function') + assert.equal(typeof provider.transcriptionModel, 'function') + assert.equal(typeof provider.speechModel, 'function') + assert.equal(typeof provider.files, 'function') +}) + +test('native transcription and speech models use the local QVAC audio endpoints', async () => { + const calls: Array<{ url: string; init?: RequestInit }> = [] + const wavBytes = Uint8Array.from([ + 82, 73, 70, 70, 36, 0, 0, 0, 87, 65, 86, 69, 102, 109, 116, 32, 16, 0, 0, 0, 1, 0, 1, 0, 128, + 62, 0, 0, 0, 125, 0, 0, 2, 0, 16, 0, 100, 97, 116, 97, 0, 0, 0, 0 + ]) + const customFetch: typeof fetch = (input, init) => { + const url = + typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url + calls.push({ url, ...(init !== undefined && { init }) }) + if (url.endsWith('/v1/audio/transcriptions')) { + assert.ok(init?.body instanceof FormData) + assert.equal(new Headers(init.headers).get('content-type'), null) + assert.equal(init.body.get('model'), 'whisper-local') + assert.equal(init.body.get('prompt'), 'Sensor vocabulary') + return Promise.resolve(Response.json({ text: 'temperature is twenty four degrees' })) + } + if (url.endsWith('/v1/audio/speech')) { + assert.equal(new Headers(init?.headers).get('content-type'), 'application/json') + const body = JSON.parse(String(init?.body)) as Record + assert.deepEqual(body, { + model: 'tts-local', + input: 'Local speech check.', + voice: 'default', + response_format: 'wav' + }) + return Promise.resolve(new Response(wavBytes, { headers: { 'content-type': 'audio/wav' } })) + } + return Promise.resolve(new Response('not found', { status: 404 })) + } + + const provider = createQvac({ + baseURL: 'http://127.0.0.1:55555/v1', + headers: { 'Content-Type': 'application/x-invalid-global-default' }, + fetch: customFetch + }) + const { generateSpeech, transcribe } = await import('ai') + const transcript = await transcribe({ + model: provider.transcriptionModel('whisper-local'), + audio: wavBytes, + providerOptions: { qvac: { prompt: 'Sensor vocabulary' } } + }) + assert.equal(transcript.text, 'temperature is twenty four degrees') + + const speech = await generateSpeech({ + model: provider.speechModel('tts-local'), + text: 'Local speech check.', + voice: 'default', + outputFormat: 'wav' + }) + assert.deepEqual(speech.audio.uint8Array, wavBytes) + assert.deepEqual( + calls.map(({ url }) => url), + ['http://127.0.0.1:55555/v1/audio/transcriptions', 'http://127.0.0.1:55555/v1/audio/speech'] + ) +}) + +test('uploadFile returns a qvac reference and language models resolve it through the local serve', async () => { + const calls: string[] = [] + let chatBody: Record | undefined + const imageBytes = Uint8Array.from([137, 80, 78, 71, 13, 10, 26, 10]) + const customFetch: typeof fetch = (input, init) => { + const url = + typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url + calls.push(url) + if (url.endsWith('/v1/files') && init?.method === 'POST') { + assert.ok(init.body instanceof FormData) + assert.equal(new Headers(init.headers).get('content-type'), null) + assert.equal(new Headers(init.headers).get('authorization'), `Bearer ${DEFAULT_API_KEY}`) + return Promise.resolve(Response.json({ id: 'file-local-1', filename: 'sensor.png' })) + } + if (url.endsWith('/v1/files/file-local-1/content')) { + const headers = new Headers(init?.headers) + assert.equal(headers.get('authorization'), 'Bearer per-call') + assert.equal(headers.get('x-configured'), 'provider') + assert.equal(headers.get('x-trace-id'), 'turn-1') + return Promise.resolve(new Response(imageBytes, { headers: { 'content-type': 'image/png' } })) + } + if (url.endsWith('/v1/chat/completions')) { + const headers = new Headers(init?.headers) + assert.equal(headers.get('authorization'), 'Bearer per-call') + assert.equal(headers.get('x-configured'), 'provider') + assert.equal(headers.get('x-trace-id'), 'turn-1') + chatBody = JSON.parse(String(init?.body)) as Record + return Promise.resolve( + Response.json({ + id: 'cmpl-file', + object: 'chat.completion', + created: 0, + model: 'vision', + choices: [ + { + index: 0, + message: { role: 'assistant', content: '24 degrees' }, + finish_reason: 'stop' + } + ], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 } + }) + ) + } + return Promise.resolve(new Response('not found', { status: 404 })) + } + const provider = createQvac({ + baseURL: 'http://127.0.0.1:55555/v1', + headers: { + 'Content-Type': 'application/x-invalid-global-default', + 'x-configured': 'provider' + }, + fetch: customFetch + }) + const { generateText, uploadFile } = await import('ai') + const uploaded = await uploadFile({ + api: provider, + data: imageBytes, + mediaType: 'image/png', + filename: 'sensor.png' + }) + assert.deepEqual(uploaded.providerReference, { qvac: 'file-local-1' }) + + const result = await generateText({ + model: provider('vision'), + headers: { authorization: 'Bearer per-call', 'x-trace-id': 'turn-1' }, + messages: [ + { + role: 'user', + content: [ + { type: 'text', text: 'Read this sensor.' }, + { type: 'file', mediaType: 'image/png', data: uploaded.providerReference } + ] + } + ] + }) + assert.equal(result.text, '24 degrees') + assert.deepEqual(calls, [ + 'http://127.0.0.1:55555/v1/files', + 'http://127.0.0.1:55555/v1/files/file-local-1/content', + 'http://127.0.0.1:55555/v1/chat/completions' + ]) + assert.match(JSON.stringify(chatBody), /data:image\/png;base64,iVBORw0KGgo=/) +}) + +test('native HTTP adapters expose structured APICallError details and retryability', async () => { + const seen = new Map() + const customFetch: typeof fetch = (input, init) => { + const url = + typeof input === 'string' ? input : input instanceof URL ? input.toString() : input.url + seen.set(url, init) + if (url.endsWith('/audio/transcriptions')) { + return Promise.resolve( + new Response('transcription busy', { + status: 429, + statusText: 'Too Many Requests', + headers: { 'x-qvac-error': 'transcription' } + }) + ) + } + if (url.endsWith('/audio/speech')) { + return Promise.resolve( + new Response('invalid speech request', { + status: 400, + statusText: 'Bad Request', + headers: { 'x-qvac-error': 'speech' } + }) + ) + } + if (url.endsWith('/files') && init?.method === 'POST') { + return Promise.resolve( + new Response('file store unavailable', { + status: 503, + statusText: 'Service Unavailable', + headers: { 'x-qvac-error': 'upload' } + }) + ) + } + if (url.endsWith('/files/file-missing/content')) { + return Promise.resolve( + new Response('file missing', { + status: 404, + statusText: 'Not Found', + headers: { 'x-qvac-error': 'reference' } + }) + ) + } + return Promise.resolve(new Response('unexpected request', { status: 500 })) + } + + const provider = createQvac({ + baseURL: 'http://127.0.0.1:55555/v1', + headers: { authorization: 'Bearer local', 'x-configured': 'yes' }, + fetch: customFetch + }) + + const transcriptionAbort = new AbortController() + const transcriptionError = await captureApiCallError( + provider.transcriptionModel('whisper-local').doGenerate({ + audio: Uint8Array.from([1, 2, 3]), + mediaType: 'audio/wav', + headers: { 'x-per-call': 'transcription' }, + abortSignal: transcriptionAbort.signal + }) + ) + assert.equal(transcriptionError.url, 'http://127.0.0.1:55555/v1/audio/transcriptions') + assert.equal(transcriptionError.statusCode, 429) + assert.equal(transcriptionError.responseBody, 'transcription busy') + assert.equal(transcriptionError.responseHeaders?.['x-qvac-error'], 'transcription') + assert.equal(transcriptionError.isRetryable, true) + assert.equal( + (transcriptionError.requestBodyValues as Record)['model'], + 'whisper-local' + ) + const transcriptionInit = seen.get(transcriptionError.url) + assert.equal(new Headers(transcriptionInit?.headers).get('x-configured'), 'yes') + assert.equal(new Headers(transcriptionInit?.headers).get('x-per-call'), 'transcription') + assert.equal(transcriptionInit?.signal, transcriptionAbort.signal) + + const speechError = await captureApiCallError( + provider.speechModel('tts-local').doGenerate({ + text: 'hello', + headers: { 'x-per-call': 'speech' } + }) + ) + assert.equal(speechError.url, 'http://127.0.0.1:55555/v1/audio/speech') + assert.equal(speechError.statusCode, 400) + assert.equal(speechError.responseBody, 'invalid speech request') + assert.equal(speechError.responseHeaders?.['x-qvac-error'], 'speech') + assert.equal(speechError.isRetryable, false) + assert.deepEqual(speechError.requestBodyValues, { model: 'tts-local', input: 'hello' }) + + const uploadError = await captureApiCallError( + provider.files().uploadFile({ + data: { type: 'data', data: Uint8Array.from([4, 5, 6]) }, + mediaType: 'application/octet-stream', + filename: 'payload.bin' + }) + ) + assert.equal(uploadError.url, 'http://127.0.0.1:55555/v1/files') + assert.equal(uploadError.statusCode, 503) + assert.equal(uploadError.responseBody, 'file store unavailable') + assert.equal(uploadError.responseHeaders?.['x-qvac-error'], 'upload') + assert.equal(uploadError.isRetryable, true) + assert.ok('file' in (uploadError.requestBodyValues as Record)) + + const referenceError = await captureApiCallError( + provider('vision').doGenerate({ + prompt: [ + { + role: 'user', + content: [ + { + type: 'file', + mediaType: 'image/png', + data: { type: 'reference', reference: { qvac: 'file-missing' } } + } + ] + } + ] + }) + ) + assert.equal(referenceError.url, 'http://127.0.0.1:55555/v1/files/file-missing/content') + assert.equal(referenceError.statusCode, 404) + assert.equal(referenceError.responseBody, 'file missing') + assert.equal(referenceError.responseHeaders?.['x-qvac-error'], 'reference') + assert.equal(referenceError.isRetryable, false) + assert.deepEqual(referenceError.requestBodyValues, {}) }) test('createQvac default instance is constructable with no options', () => { @@ -88,6 +371,48 @@ test('createQvac forwards baseURL/apiKey/headers/fetch to the underlying call', assert.equal(capturedCustomHeader, 'flowed-through', 'custom headers should propagate') }) +test('caller authorization headers override apiKey without duplicate casing', async () => { + const capturedAuth: string[] = [] + const customFetch: typeof fetch = (input, init) => { + const headers = new Headers( + init?.headers ?? (input instanceof Request ? input.headers : undefined) + ) + capturedAuth.push(headers.get('authorization') ?? '') + return Promise.resolve( + Response.json({ + id: 'cmpl-auth', + object: 'chat.completion', + created: 0, + model: 'test-model', + choices: [ + { + index: 0, + message: { role: 'assistant', content: 'ok' }, + finish_reason: 'stop' + } + ], + usage: { prompt_tokens: 1, completion_tokens: 1, total_tokens: 2 } + }) + ) + } + const { generateText } = await import('ai') + + for (const [name, value] of [ + ['authorization', 'Bearer lowercase'], + ['Authorization', 'Bearer uppercase'] + ] as const) { + const provider = createQvac({ + baseURL: 'http://127.0.0.1:55555/v1', + apiKey: 'must-not-override-caller', + headers: { [name]: value }, + fetch: customFetch + }) + await generateText({ model: provider('test-model'), prompt: 'hi' }) + } + + assert.deepEqual(capturedAuth, ['Bearer lowercase', 'Bearer uppercase']) +}) + test('createQvac without explicit baseURL uses DEFAULT_BASE_URL', async () => { let capturedUrl: string | undefined // lunte-disable-next-line require-await diff --git a/packages/cli/src/serve/routes/files.ts b/packages/cli/src/serve/routes/files.ts index 3762ab4a50..8b6dfd547c 100644 --- a/packages/cli/src/serve/routes/files.ts +++ b/packages/cli/src/serve/routes/files.ts @@ -76,7 +76,8 @@ const plugin: FastifyPluginAsyncZod = async (app) => { const id = ctx.ephemeralFiles.put({ data: fileBuf, fileName: fileMeta.filename.length > 0 ? fileMeta.filename : 'upload.bin', - purpose + purpose, + contentType: fileMeta.mimetype }) ctx.logger.info(` files upload id=${id} bytes=${fileBuf.length} purpose=${purpose}`) const rec = ctx.ephemeralFiles.get(id) diff --git a/packages/cli/test/e2e/http/files-validation.test.ts b/packages/cli/test/e2e/http/files-validation.test.ts index 4b3a8b5b84..656418771a 100644 --- a/packages/cli/test/e2e/http/files-validation.test.ts +++ b/packages/cli/test/e2e/http/files-validation.test.ts @@ -28,6 +28,7 @@ describe('serve: files content endpoint', () => { const res = await server().inject({ method: 'GET', url: `/v1/files/${id}/content` }) assert.equal(res.statusCode, 200) + assert.equal(res.headers['content-type'], 'image/png') assert.ok(res.rawPayload.equals(png), 'downloaded bytes should match the upload') })