feat(providers/lmnt): add speech (#5726)

.changeset/modern-kings-smoke.md

@@ -0,0 +1,5 @@
+---
+'@ai-sdk/lmnt': patch
+---
+
+feat(providers/lmnt): add speech

content/docs/02-foundations/02-providers-and-models.mdx

@@ -41,6 +41,7 @@ The AI SDK comes with a wide range of providers that you can use to interact wit
 - [Groq Provider](/providers/ai-sdk-providers/groq) (`@ai-sdk/groq`)
 - [Perplexity Provider](/providers/ai-sdk-providers/perplexity) (`@ai-sdk/perplexity`)
 - [ElevenLabs Provider](/providers/ai-sdk-providers/elevenlabs) (`@ai-sdk/elevenlabs`)
+- [LMNT Provider](/providers/ai-sdk-providers/lmnt) (`@ai-sdk/lmnt`)
 - [Hume Provider](/providers/ai-sdk-providers/hume) (`@ai-sdk/hume`)
 - [Rev.ai Provider](/providers/ai-sdk-providers/revai) (`@ai-sdk/revai`)
 

content/docs/03-ai-sdk-core/37-speech.mdx

@@ -145,6 +145,8 @@ try {
 | [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `tts-1`           |
 | [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `tts-1-hd`        |
 | [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `gpt-4o-mini-tts` |
+| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models)     | `aurora`          |
+| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models)     | `blizzard`        |
 | [Hume](/providers/ai-sdk-providers/hume#speech-models)     | `default`         |
 
 Above are a small subset of the speech models supported by the AI SDK providers. For more, see the respective provider documentation.

content/providers/01-ai-sdk-providers/140-lmnt.mdx

@@ -0,0 +1,136 @@
+---
+title: LMNT
+description: Learn how to use the LMNT provider for the AI SDK.
+---
+
+# LMNT Provider
+
+The [LMNT](https://lmnt.com/) provider contains language model support for the LMNT transcription API.
+
+## Setup
+
+The LMNT provider is available in the `@ai-sdk/lmnt` module. You can install it with
+
+<Tabs items={['pnpm', 'npm', 'yarn']}>
+  <Tab>
+    <Snippet text="pnpm add @ai-sdk/lmnt" dark />
+  </Tab>
+  <Tab>
+    <Snippet text="npm install @ai-sdk/lmnt" dark />
+  </Tab>
+  <Tab>
+    <Snippet text="yarn add @ai-sdk/lmnt" dark />
+  </Tab>
+</Tabs>
+
+## Provider Instance
+
+You can import the default provider instance `lmnt` from `@ai-sdk/lmnt`:
+
+```ts
+import { lmnt } from '@ai-sdk/lmnt';
+```
+
+If you need a customized setup, you can import `createLMNT` from `@ai-sdk/lmnt` and create a provider instance with your settings:
+
+```ts
+import { createLMNT } from '@ai-sdk/lmnt';
+
+const lmnt = createLMNT({
+  // custom settings, e.g.
+  fetch: customFetch,
+});
+```
+
+You can use the following optional settings to customize the LMNT provider instance:
+
+- **apiKey** _string_
+
+  API key that is being sent using the `Authorization` header.
+  It defaults to the `LMNT_API_KEY` environment variable.
+
+- **headers** _Record&lt;string,string&gt;_
+
+  Custom headers to include in the requests.
+
+- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_
+
+  Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
+  Defaults to the global `fetch` function.
+  You can use it as a middleware to intercept requests,
+  or to provide a custom fetch implementation for e.g. testing.
+
+## Speech Models
+
+You can create models that call the [LMNT speech API](https://docs.lmnt.com/api-reference/speech/synthesize-speech-bytes)
+using the `.speech()` factory method.
+
+The first argument is the model id e.g. `aurora`.
+
+```ts
+const model = lmnt.speech('aurora');
+```
+
+You can also pass additional provider-specific options using the `providerOptions` argument. For example, supplying a voice to use for the generated audio.
+
+```ts highlight="6"
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import { lmnt } from '@ai-sdk/lmnt';
+
+const result = await generateSpeech({
+  model: lmnt.speech('aurora'),
+  text: 'Hello, world!',
+  providerOptions: { lmnt: { language: 'en' } },
+});
+```
+
+### Provider Options
+
+The LMNT provider accepts the following options:
+
+- **model** _'aurora' | 'blizzard'_
+
+  The LMNT model to use. Defaults to `'aurora'`.
+
+- **language** _'auto' | 'en' | 'es' | 'pt' | 'fr' | 'de' | 'zh' | 'ko' | 'hi' | 'ja' | 'ru' | 'it' | 'tr'_
+
+  The language to use for speech synthesis. Defaults to `'auto'`.
+
+- **format** _'aac' | 'mp3' | 'mulaw' | 'raw' | 'wav'_
+
+  The audio format to return. Defaults to `'mp3'`.
+
+- **sampleRate** _number_
+
+  The sample rate of the audio in Hz. Defaults to `24000`.
+
+- **speed** _number_
+
+  The speed of the speech. Must be between 0.25 and 2. Defaults to `1`.
+
+- **seed** _number_
+
+  An optional seed for deterministic generation.
+
+- **conversational** _boolean_
+
+  Whether to use a conversational style. Defaults to `false`.
+
+- **length** _number_
+
+  Maximum length of the audio in seconds. Maximum value is 300.
+
+- **topP** _number_
+
+  Top-p sampling parameter. Must be between 0 and 1. Defaults to `1`.
+
+- **temperature** _number_
+
+  Temperature parameter for sampling. Must be at least 0. Defaults to `1`.
+
+### Model Capabilities
+
+| Model      | Instructions        |
+| ---------- | ------------------- |
+| `aurora`   | <Check size={18} /> |
+| `blizzard` | <Check size={18} /> |

examples/ai-core/package.json

@@ -16,6 +16,7 @@
     "@ai-sdk/google": "2.0.0-canary.9",
     "@ai-sdk/google-vertex": "3.0.0-canary.9",
     "@ai-sdk/groq": "2.0.0-canary.9",
+    "@ai-sdk/lmnt": "1.0.0-canary.0",
     "@ai-sdk/luma": "1.0.0-canary.8",
     "@ai-sdk/hume": "1.0.0-canary.0",
     "@ai-sdk/mistral": "2.0.0-canary.8",

examples/ai-core/src/generate-speech/lmnt-speed.ts

@@ -0,0 +1,21 @@
+import { lmnt } from '@ai-sdk/lmnt';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import 'dotenv/config';
+import { saveAudioFile } from '../lib/save-audio';
+
+async function main() {
+  const result = await generateSpeech({
+    model: lmnt.speech('aurora'),
+    text: 'Hello from the AI SDK!',
+    speed: 1.5,
+  });
+
+  console.log('Audio:', result.audio);
+  console.log('Warnings:', result.warnings);
+  console.log('Responses:', result.responses);
+  console.log('Provider Metadata:', result.providerMetadata);
+
+  await saveAudioFile(result.audio);
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-speech/lmnt-voice.ts

@@ -0,0 +1,21 @@
+import { lmnt } from '@ai-sdk/lmnt';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import 'dotenv/config';
+import { saveAudioFile } from '../lib/save-audio';
+
+async function main() {
+  const result = await generateSpeech({
+    model: lmnt.speech('aurora'),
+    text: 'Hello from the AI SDK!',
+    voice: 'nova',
+  });
+
+  console.log('Audio:', result.audio);
+  console.log('Warnings:', result.warnings);
+  console.log('Responses:', result.responses);
+  console.log('Provider Metadata:', result.providerMetadata);
+
+  await saveAudioFile(result.audio);
+}
+
+main().catch(console.error);

examples/ai-core/src/generate-speech/lmnt.ts

@@ -0,0 +1,20 @@
+import { lmnt } from '@ai-sdk/lmnt';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+import 'dotenv/config';
+import { saveAudioFile } from '../lib/save-audio';
+
+async function main() {
+  const result = await generateSpeech({
+    model: lmnt.speech('aurora'),
+    text: 'Hello from the AI SDK!',
+  });
+
+  console.log('Audio:', result.audio);
+  console.log('Warnings:', result.warnings);
+  console.log('Responses:', result.responses);
+  console.log('Provider Metadata:', result.providerMetadata);
+
+  await saveAudioFile(result.audio);
+}
+
+main().catch(console.error);

examples/ai-core/tsconfig.json

@@ -49,6 +49,9 @@
     {
       "path": "../../packages/deepinfra"
     },
+    {
+      "path": "../../packages/lmnt"
+    },
     {
       "path": "../../packages/deepseek"
     },

packages/lmnt/README.md

@@ -0,0 +1,36 @@
+# AI SDK - LMNT Provider
+
+The **[LMNT provider](https://sdk.vercel.ai/providers/ai-sdk-providers/lmnt)** for the [AI SDK](https://sdk.vercel.ai/docs)
+contains language model support for the LMNT API.
+
+## Setup
+
+The LMNT provider is available in the `@ai-sdk/lmnt` module. You can install it with
+
+```bash
+npm i @ai-sdk/lmnt
+```
+
+## Provider Instance
+
+You can import the default provider instance `lmnt` from `@ai-sdk/lmnt`:
+
+```ts
+import { lmnt } from '@ai-sdk/lmnt';
+```
+
+## Example
+
+```ts
+import { lmnt } from '@ai-sdk/lmnt';
+import { experimental_generateSpeech as generateSpeech } from 'ai';
+
+const result = await generateSpeech({
+  model: lmnt.speech('aurora'),
+  text: 'Hello, world!',
+});
+```
+
+## Documentation
+
+Please check out the **[LMNT provider documentation](https://sdk.vercel.ai/providers/ai-sdk-providers/lmnt)** for more information.

packages/lmnt/package.json

@@ -0,0 +1,64 @@
+{
+  "name": "@ai-sdk/lmnt",
+  "version": "1.0.0-canary.0",
+  "license": "Apache-2.0",
+  "sideEffects": false,
+  "main": "./dist/index.js",
+  "module": "./dist/index.mjs",
+  "types": "./dist/index.d.ts",
+  "files": [
+    "dist/**/*",
+    "CHANGELOG.md"
+  ],
+  "scripts": {
+    "build": "tsup --tsconfig tsconfig.build.json",
+    "build:watch": "tsup --tsconfig tsconfig.build.json --watch",
+    "clean": "rm -rf dist",
+    "lint": "eslint \"./**/*.ts*\"",
+    "type-check": "tsc --noEmit",
+    "prettier-check": "prettier --check \"./**/*.ts*\"",
+    "test": "pnpm test:node && pnpm test:edge",
+    "test:edge": "vitest --config vitest.edge.config.js --run",
+    "test:node": "vitest --config vitest.node.config.js --run",
+    "test:node:watch": "vitest --config vitest.node.config.js --watch"
+  },
+  "exports": {
+    "./package.json": "./package.json",
+    ".": {
+      "types": "./dist/index.d.ts",
+      "import": "./dist/index.mjs",
+      "require": "./dist/index.js"
+    }
+  },
+  "dependencies": {
+    "@ai-sdk/provider": "2.0.0-canary.7",
+    "@ai-sdk/provider-utils": "3.0.0-canary.8"
+  },
+  "devDependencies": {
+    "@types/node": "20.17.24",
+    "@vercel/ai-tsconfig": "workspace:*",
+    "tsup": "^8",
+    "typescript": "5.6.3",
+    "zod": "3.23.8"
+  },
+  "peerDependencies": {
+    "zod": "^3.0.0"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "homepage": "https://sdk.vercel.ai/docs",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/vercel/ai.git"
+  },
+  "bugs": {
+    "url": "https://github.com/vercel/ai/issues"
+  },
+  "keywords": [
+    "ai"
+  ]
+}

packages/lmnt/src/index.ts

@@ -0,0 +1,2 @@
+export { createLMNT, lmnt } from './lmnt-provider';
+export type { LMNTProvider, LMNTProviderSettings } from './lmnt-provider';

packages/lmnt/src/lmnt-api-types.ts

@@ -0,0 +1,39 @@
+export type LMNTSpeechAPITypes = {
+  /** The voice id of the voice to use; voice ids can be retrieved by calls to List voices or Voice info. */
+  voice: string;
+  /** The text to synthesize; max 5000 characters per request (including spaces) */
+  text: string;
+  /** The model to use for synthesis. One of aurora (default) or blizzard. */
+  model?: 'aurora' | 'blizzard';
+  /** The desired language. Two letter ISO 639-1 code. Does not work with professional clones. Not all languages work with all models. Defaults to auto language detection. */
+  language?:
+    | 'auto'
+    | 'en'
+    | 'es'
+    | 'pt'
+    | 'fr'
+    | 'de'
+    | 'zh'
+    | 'ko'
+    | 'hi'
+    | 'ja'
+    | 'ru'
+    | 'it'
+    | 'tr';
+  /** The file format of the audio output */
+  format?: 'aac' | 'mp3' | 'mulaw' | 'raw' | 'wav';
+  /** The desired output sample rate in Hz */
+  sample_rate?: 8000 | 16000 | 24000;
+  /** The talking speed of the generated speech, a floating point value between 0.25 (slow) and 2.0 (fast). */
+  speed?: number;
+  /** Seed used to specify a different take; defaults to random */
+  seed?: number;
+  /** Set this to true to generate conversational-style speech rather than reading-style speech. Does not work with the blizzard model. */
+  conversational?: boolean;
+  /** Produce speech of this length in seconds; maximum 300.0 (5 minutes). Does not work with the blizzard model. */
+  length?: number;
+  /** Controls the stability of the generated speech. A lower value (like 0.3) produces more consistent, reliable speech. A higher value (like 0.9) gives more flexibility in how words are spoken, but might occasionally produce unusual intonations or speech patterns. */
+  top_p?: number;
+  /** Influences how expressive and emotionally varied the speech becomes. Lower values (like 0.3) create more neutral, consistent speaking styles. Higher values (like 1.0) allow for more dynamic emotional range and speaking styles. */
+  temperature?: number;
+};

packages/lmnt/src/lmnt-config.ts

@@ -0,0 +1,9 @@
+import { FetchFunction } from '@ai-sdk/provider-utils';
+
+export type LMNTConfig = {
+  provider: string;
+  url: (options: { modelId: string; path: string }) => string;
+  headers: () => Record<string, string | undefined>;
+  fetch?: FetchFunction;
+  generateId?: () => string;
+};