Skip to content

Commit 40d6708

Browse files
samdentyhaydenbleaselCopilotlgrammel
authored
feat(providers/lmnt): add speech (#5726) (#5823)
Co-authored-by: Hayden Bleasel <[email protected]> Co-authored-by: Copilot <[email protected]> Co-authored-by: Lars Grammel <[email protected]>
1 parent 018f3c7 commit 40d6708

28 files changed

+985
-0
lines changed

‎.changeset/modern-kings-smoke.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@ai-sdk/lmnt': patch
3+
---
4+
5+
feat(providers/lmnt): add speech

‎content/docs/02-foundations/02-providers-and-models.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,7 @@ The AI SDK comes with a wide range of providers that you can use to interact wit
4141
- [Groq Provider](/providers/ai-sdk-providers/groq) (`@ai-sdk/groq`)
4242
- [Perplexity Provider](/providers/ai-sdk-providers/perplexity) (`@ai-sdk/perplexity`)
4343
- [ElevenLabs Provider](/providers/ai-sdk-providers/elevenlabs) (`@ai-sdk/elevenlabs`)
44+
- [LMNT Provider](/providers/ai-sdk-providers/lmnt) (`@ai-sdk/lmnt`)
4445
- [Hume Provider](/providers/ai-sdk-providers/hume) (`@ai-sdk/hume`)
4546
- [Rev.ai Provider](/providers/ai-sdk-providers/revai) (`@ai-sdk/revai`)
4647

‎content/docs/03-ai-sdk-core/37-speech.mdx‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,8 @@ try {
145145
| [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `tts-1` |
146146
| [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `tts-1-hd` |
147147
| [OpenAI](/providers/ai-sdk-providers/openai#speech-models) | `gpt-4o-mini-tts` |
148+
| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models) | `aurora` |
149+
| [LMNT](/providers/ai-sdk-providers/lmnt#speech-models) | `blizzard` |
148150
| [Hume](/providers/ai-sdk-providers/hume#speech-models) | `default` |
149151

150152
Above are a small subset of the speech models supported by the AI SDK providers. For more, see the respective provider documentation.

‎content/providers/01-ai-sdk-providers/140-lmnt.mdx‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
---
2+
title: LMNT
3+
description: Learn how to use the LMNT provider for the AI SDK.
4+
---
5+
6+
# LMNT Provider
7+
8+
The [LMNT](https://lmnt.com/) provider contains language model support for the LMNT transcription API.
9+
10+
## Setup
11+
12+
The LMNT provider is available in the `@ai-sdk/lmnt` module. You can install it with
13+
14+
<Tabs items={['pnpm', 'npm', 'yarn']}>
15+
<Tab>
16+
<Snippet text="pnpm add @ai-sdk/lmnt" dark />
17+
</Tab>
18+
<Tab>
19+
<Snippet text="npm install @ai-sdk/lmnt" dark />
20+
</Tab>
21+
<Tab>
22+
<Snippet text="yarn add @ai-sdk/lmnt" dark />
23+
</Tab>
24+
</Tabs>
25+
26+
## Provider Instance
27+
28+
You can import the default provider instance `lmnt` from `@ai-sdk/lmnt`:
29+
30+
```ts
31+
import { lmnt } from '@ai-sdk/lmnt';
32+
```
33+
34+
If you need a customized setup, you can import `createLMNT` from `@ai-sdk/lmnt` and create a provider instance with your settings:
35+
36+
```ts
37+
import { createLMNT } from '@ai-sdk/lmnt';
38+
39+
const lmnt = createLMNT({
40+
// custom settings, e.g.
41+
fetch: customFetch,
42+
});
43+
```
44+
45+
You can use the following optional settings to customize the LMNT provider instance:
46+
47+
- **apiKey** _string_
48+
49+
API key that is being sent using the `Authorization` header.
50+
It defaults to the `LMNT_API_KEY` environment variable.
51+
52+
- **headers** _Record&lt;string,string&gt;_
53+
54+
Custom headers to include in the requests.
55+
56+
- **fetch** _(input: RequestInfo, init?: RequestInit) => Promise&lt;Response&gt;_
57+
58+
Custom [fetch](https://developer.mozilla.org/en-US/docs/Web/API/fetch) implementation.
59+
Defaults to the global `fetch` function.
60+
You can use it as a middleware to intercept requests,
61+
or to provide a custom fetch implementation for e.g. testing.
62+
63+
## Speech Models
64+
65+
You can create models that call the [LMNT speech API](https://docs.lmnt.com/api-reference/speech/synthesize-speech-bytes)
66+
using the `.speech()` factory method.
67+
68+
The first argument is the model id e.g. `aurora`.
69+
70+
```ts
71+
const model = lmnt.speech('aurora');
72+
```
73+
74+
You can also pass additional provider-specific options using the `providerOptions` argument. For example, supplying a voice to use for the generated audio.
75+
76+
```ts highlight="6"
77+
import { experimental_generateSpeech as generateSpeech } from 'ai';
78+
import { lmnt } from '@ai-sdk/lmnt';
79+
80+
const result = await generateSpeech({
81+
model: lmnt.speech('aurora'),
82+
text: 'Hello, world!',
83+
providerOptions: { lmnt: { language: 'en' } },
84+
});
85+
```
86+
87+
### Provider Options
88+
89+
The LMNT provider accepts the following options:
90+
91+
- **model** _'aurora' | 'blizzard'_
92+
93+
The LMNT model to use. Defaults to `'aurora'`.
94+
95+
- **language** _'auto' | 'en' | 'es' | 'pt' | 'fr' | 'de' | 'zh' | 'ko' | 'hi' | 'ja' | 'ru' | 'it' | 'tr'_
96+
97+
The language to use for speech synthesis. Defaults to `'auto'`.
98+
99+
- **format** _'aac' | 'mp3' | 'mulaw' | 'raw' | 'wav'_
100+
101+
The audio format to return. Defaults to `'mp3'`.
102+
103+
- **sampleRate** _number_
104+
105+
The sample rate of the audio in Hz. Defaults to `24000`.
106+
107+
- **speed** _number_
108+
109+
The speed of the speech. Must be between 0.25 and 2. Defaults to `1`.
110+
111+
- **seed** _number_
112+
113+
An optional seed for deterministic generation.
114+
115+
- **conversational** _boolean_
116+
117+
Whether to use a conversational style. Defaults to `false`.
118+
119+
- **length** _number_
120+
121+
Maximum length of the audio in seconds. Maximum value is 300.
122+
123+
- **topP** _number_
124+
125+
Top-p sampling parameter. Must be between 0 and 1. Defaults to `1`.
126+
127+
- **temperature** _number_
128+
129+
Temperature parameter for sampling. Must be at least 0. Defaults to `1`.
130+
131+
### Model Capabilities
132+
133+
| Model | Instructions |
134+
| ---------- | ------------------- |
135+
| `aurora` | <Check size={18} /> |
136+
| `blizzard` | <Check size={18} /> |

‎examples/ai-core/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@
1616
"@ai-sdk/google": "2.0.0-canary.9",
1717
"@ai-sdk/google-vertex": "3.0.0-canary.9",
1818
"@ai-sdk/groq": "2.0.0-canary.9",
19+
"@ai-sdk/lmnt": "1.0.0-canary.0",
1920
"@ai-sdk/luma": "1.0.0-canary.8",
2021
"@ai-sdk/hume": "1.0.0-canary.0",
2122
"@ai-sdk/mistral": "2.0.0-canary.8",

‎examples/ai-core/src/generate-speech/lmnt-speed.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
import { lmnt } from '@ai-sdk/lmnt';
2+
import { experimental_generateSpeech as generateSpeech } from 'ai';
3+
import 'dotenv/config';
4+
import { saveAudioFile } from '../lib/save-audio';
5+
6+
async function main() {
7+
const result = await generateSpeech({
8+
model: lmnt.speech('aurora'),
9+
text: 'Hello from the AI SDK!',
10+
speed: 1.5,
11+
});
12+
13+
console.log('Audio:', result.audio);
14+
console.log('Warnings:', result.warnings);
15+
console.log('Responses:', result.responses);
16+
console.log('Provider Metadata:', result.providerMetadata);
17+
18+
await saveAudioFile(result.audio);
19+
}
20+
21+
main().catch(console.error);

‎examples/ai-core/src/generate-speech/lmnt-voice.ts‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
import { lmnt } from '@ai-sdk/lmnt';
2+
import { experimental_generateSpeech as generateSpeech } from 'ai';
3+
import 'dotenv/config';
4+
import { saveAudioFile } from '../lib/save-audio';
5+
6+
async function main() {
7+
const result = await generateSpeech({
8+
model: lmnt.speech('aurora'),
9+
text: 'Hello from the AI SDK!',
10+
voice: 'nova',
11+
});
12+
13+
console.log('Audio:', result.audio);
14+
console.log('Warnings:', result.warnings);
15+
console.log('Responses:', result.responses);
16+
console.log('Provider Metadata:', result.providerMetadata);
17+
18+
await saveAudioFile(result.audio);
19+
}
20+
21+
main().catch(console.error);

‎examples/ai-core/src/generate-speech/lmnt.ts‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
import { lmnt } from '@ai-sdk/lmnt';
2+
import { experimental_generateSpeech as generateSpeech } from 'ai';
3+
import 'dotenv/config';
4+
import { saveAudioFile } from '../lib/save-audio';
5+
6+
async function main() {
7+
const result = await generateSpeech({
8+
model: lmnt.speech('aurora'),
9+
text: 'Hello from the AI SDK!',
10+
});
11+
12+
console.log('Audio:', result.audio);
13+
console.log('Warnings:', result.warnings);
14+
console.log('Responses:', result.responses);
15+
console.log('Provider Metadata:', result.providerMetadata);
16+
17+
await saveAudioFile(result.audio);
18+
}
19+
20+
main().catch(console.error);

‎examples/ai-core/tsconfig.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,9 @@
4949
{
5050
"path": "../../packages/deepinfra"
5151
},
52+
{
53+
"path": "../../packages/lmnt"
54+
},
5255
{
5356
"path": "../../packages/deepseek"
5457
},

‎packages/lmnt/README.md‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# AI SDK - LMNT Provider
2+
3+
The **[LMNT provider](https://sdk.vercel.ai/providers/ai-sdk-providers/lmnt)** for the [AI SDK](https://sdk.vercel.ai/docs)
4+
contains language model support for the LMNT API.
5+
6+
## Setup
7+
8+
The LMNT provider is available in the `@ai-sdk/lmnt` module. You can install it with
9+
10+
```bash
11+
npm i @ai-sdk/lmnt
12+
```
13+
14+
## Provider Instance
15+
16+
You can import the default provider instance `lmnt` from `@ai-sdk/lmnt`:
17+
18+
```ts
19+
import { lmnt } from '@ai-sdk/lmnt';
20+
```
21+
22+
## Example
23+
24+
```ts
25+
import { lmnt } from '@ai-sdk/lmnt';
26+
import { experimental_generateSpeech as generateSpeech } from 'ai';
27+
28+
const result = await generateSpeech({
29+
model: lmnt.speech('aurora'),
30+
text: 'Hello, world!',
31+
});
32+
```
33+
34+
## Documentation
35+
36+
Please check out the **[LMNT provider documentation](https://sdk.vercel.ai/providers/ai-sdk-providers/lmnt)** for more information.

0 commit comments

Comments
 (0)