Skip to content

Commit 91e6c86

Browse files
github-actions[bot]github-actions[bot]
committed
docs: add Smallest AI STT service documentation
Add documentation for SmallestSTTService introduced in pipecat PR #4162. - Created server/services/stt/smallest.mdx with full configuration and usage - Added to docs.json navigation (Speech-to-Text section) - Added to supported-services.mdx table The new service provides real-time speech-to-text using Smallest AI's Pulse WebSocket API with support for multiple languages, diarization, PII redaction, and other advanced transcription features.
1 parent 69a8ea7 commit 91e6c86

3 files changed

Lines changed: 165 additions & 0 deletions

File tree

‎docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,7 @@
149149
"server/services/stt/nvidia",
150150
"server/services/stt/openai",
151151
"server/services/stt/sarvam",
152+
"server/services/stt/smallest",
152153
"server/services/stt/soniox",
153154
"server/services/stt/speechmatics",
154155
"server/services/stt/whisper"

‎server/services/stt/smallest.mdx‎

Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
---
2+
title: "Smallest AI"
3+
description: "Speech-to-text service using Smallest AI's Pulse WebSocket API"
4+
---
5+
6+
## Overview
7+
8+
Smallest AI provides real-time speech-to-text transcription through a WebSocket-based integration with their Waves API. The service uses the Pulse model to stream audio continuously and receive interim and final transcription results with low latency.
9+
10+
<CardGroup cols={2}>
11+
<Card
12+
title="Smallest AI STT API Reference"
13+
icon="code"
14+
href="https://reference-server.pipecat.ai/en/latest/api/pipecat.services.smallest.stt.html"
15+
>
16+
Complete API reference for all parameters and methods
17+
</Card>
18+
<Card
19+
title="Example Implementation"
20+
icon="play"
21+
href="https://github.com/pipecat-ai/pipecat/blob/main/examples/voice/voice-smallest.py"
22+
>
23+
Complete example with WebSocket streaming
24+
</Card>
25+
</CardGroup>
26+
27+
## Installation
28+
29+
```bash
30+
pip install "pipecat-ai[smallest]"
31+
```
32+
33+
## Prerequisites
34+
35+
1. **Smallest AI Account**: Sign up at [Smallest AI](https://www.smallest.ai/)
36+
2. **API Key**: Generate an API key from your account dashboard
37+
38+
Set the following environment variable:
39+
40+
```bash
41+
export SMALLEST_API_KEY=your_api_key
42+
```
43+
44+
## Configuration
45+
46+
<ParamField path="api_key" type="str" required>
47+
Smallest AI API key for authentication.
48+
</ParamField>
49+
50+
<ParamField path="base_url" type="str" default="wss://api.smallest.ai">
51+
Base WebSocket URL for the Smallest API. Override for custom or proxied
52+
deployments.
53+
</ParamField>
54+
55+
<ParamField path="encoding" type="str" default="linear16">
56+
Audio encoding format.
57+
</ParamField>
58+
59+
<ParamField path="sample_rate" type="int" default="None">
60+
Audio sample rate in Hz. When `None`, uses the pipeline's configured sample
61+
rate.
62+
</ParamField>
63+
64+
<ParamField path="settings" type="SmallestSTTService.Settings" default="None">
65+
Runtime-configurable settings. See [Settings](#settings) below.
66+
</ParamField>
67+
68+
<ParamField path="ttfs_p99_latency" type="float" default="SMALLEST_TTFS_P99">
69+
P99 latency from speech end to final transcript in seconds. Used for
70+
processing metrics.
71+
</ParamField>
72+
73+
### Settings
74+
75+
Runtime-configurable settings passed via the `settings` constructor argument using `SmallestSTTService.Settings(...)`. These can be updated mid-conversation with `STTUpdateSettingsFrame`. See [Service Settings](/guides/fundamentals/service-settings) for details.
76+
77+
| Parameter | Type | Default | Description |
78+
| --------------------- | ----------------- | ------------- | ------------------------------------------------------------------------ |
79+
| `model` | `str` | `pulse` | Model identifier. Currently only `pulse` is supported. |
80+
| `language` | `Language \| str` | `Language.EN` | Language code for transcription. |
81+
| `word_timestamps` | `bool` | `False` | Include word-level timestamps in transcription results. |
82+
| `full_transcript` | `bool` | `False` | Include cumulative transcript in results. |
83+
| `sentence_timestamps` | `bool` | `False` | Include sentence-level timestamps in transcription results. |
84+
| `redact_pii` | `bool` | `False` | Redact personally identifiable information from transcripts. |
85+
| `redact_pci` | `bool` | `False` | Redact payment card information from transcripts. |
86+
| `numerals` | `str` | `auto` | Convert spoken numerals to digits. Options: `auto`, `always`, or `none`. |
87+
| `diarize` | `bool` | `False` | Enable speaker diarization to identify different speakers. |
88+
89+
## Usage
90+
91+
### Basic Setup
92+
93+
```python
94+
from pipecat.services.smallest.stt import SmallestSTTService
95+
from pipecat.transcriptions.language import Language
96+
97+
stt = SmallestSTTService(
98+
api_key=os.getenv("SMALLEST_API_KEY"),
99+
settings=SmallestSTTService.Settings(
100+
language=Language.EN,
101+
),
102+
)
103+
```
104+
105+
### With Advanced Features
106+
107+
```python
108+
stt = SmallestSTTService(
109+
api_key=os.getenv("SMALLEST_API_KEY"),
110+
settings=SmallestSTTService.Settings(
111+
language=Language.ES,
112+
word_timestamps=True,
113+
diarize=True,
114+
redact_pii=True,
115+
),
116+
)
117+
```
118+
119+
### Updating Settings at Runtime
120+
121+
Transcription settings can be changed mid-conversation using `STTUpdateSettingsFrame`:
122+
123+
```python
124+
from pipecat.frames.frames import STTUpdateSettingsFrame
125+
from pipecat.services.smallest.stt import SmallestSTTSettings
126+
127+
await task.queue_frame(
128+
STTUpdateSettingsFrame(
129+
delta=SmallestSTTSettings(
130+
language=Language.FR,
131+
word_timestamps=False,
132+
)
133+
)
134+
)
135+
```
136+
137+
<Tip>
138+
Changing settings will trigger a WebSocket reconnection, which may cause a
139+
brief interruption in transcription.
140+
</Tip>
141+
142+
## Notes
143+
144+
- **WebSocket streaming**: The service uses WebSocket connections for real-time streaming. The connection is automatically managed and will reconnect if interrupted.
145+
- **VAD integration**: Uses Pipecat's VAD to detect when the user stops speaking and sends a finalize message to flush the final transcript.
146+
- **Keepalive**: The service sends periodic keepalive messages (every 5 seconds) to prevent idle timeouts on the WebSocket connection.
147+
- **Language support**: Supports 31 languages including Bulgarian, Bengali, Czech, Danish, German, English, Spanish, Estonian, Finnish, French, Gujarati, Hindi, Hungarian, Italian, Kannada, Lithuanian, Latvian, Malayalam, Marathi, Maltese, Dutch, Odia, Punjabi, Polish, Portuguese, Romanian, Russian, Slovak, Swedish, Tamil, Telugu, and Ukrainian.
148+
149+
## Event Handlers
150+
151+
Smallest AI STT supports the standard [service connection events](/server/events/service-events):
152+
153+
| Event | Description |
154+
| --------------------- | ------------------------------------ |
155+
| `on_connected` | Connected to Smallest AI WebSocket |
156+
| `on_disconnected` | Disconnected from Smallest WebSocket |
157+
| `on_connection_error` | WebSocket connection error occurred |
158+
159+
```python
160+
@stt.event_handler("on_connected")
161+
async def on_connected(service):
162+
print("Connected to Smallest AI STT")
163+
```

‎server/services/supported-services.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@ Speech-to-Text services receive and audio input and output transcriptions.
5252
| [NVIDIA](/server/services/stt/nvidia) | `pip install "pipecat-ai[nvidia]"` |
5353
| [OpenAI](/server/services/stt/openai) | `pip install "pipecat-ai[openai]"` |
5454
| [Sarvam](/server/services/stt/sarvam) | `pip install "pipecat-ai[sarvam]"` |
55+
| [Smallest AI](/server/services/stt/smallest) | `pip install "pipecat-ai[smallest]"` |
5556
| [Soniox](/server/services/stt/soniox) | `pip install "pipecat-ai[soniox]"` |
5657
| [Speechmatics](/server/services/stt/speechmatics) | `pip install "pipecat-ai[speechmatics]"` |
5758
| [Whisper](/server/services/stt/whisper) | `pip install "pipecat-ai[whisper]"` |

0 commit comments

Comments
 (0)