|
| 1 | +--- |
| 2 | +title: "Hermes Agent" |
| 3 | +description: "Use Voicebox as the voice and ears of Hermes Agent — spoken replies and voice-message transcription, fully local." |
| 4 | +--- |
| 5 | + |
| 6 | +## Overview |
| 7 | + |
| 8 | +[Hermes Agent](https://github.com/NousResearch/hermes-agent) is Nous |
| 9 | +Research's open-source self-improving agent: a terminal CLI/TUI plus a |
| 10 | +messaging gateway that connects one agent to Telegram, Discord, WhatsApp, |
| 11 | +Slack, and Signal. It has first-class voice features — spoken replies, |
| 12 | +voice-bubble delivery on chat platforms, push-to-talk dictation, and |
| 13 | +automatic transcription of incoming voice messages — and every one of them |
| 14 | +is pluggable. |
| 15 | + |
| 16 | +Voicebox slots into both directions of that loop, entirely on-device: |
| 17 | + |
| 18 | +- **Voice out** — Hermes speaks its replies in one of your cloned or preset |
| 19 | + voices instead of a stock cloud voice. |
| 20 | +- **Voice in** — voice messages and push-to-talk audio are transcribed by |
| 21 | + the Whisper models already bundled with Voicebox. Audio never leaves your |
| 22 | + machine. |
| 23 | + |
| 24 | +There are two integration surfaces, and they compose — most people will |
| 25 | +want both. Everything talks to the same local API |
| 26 | +(`http://127.0.0.1:17493` while the Voicebox app is running). |
| 27 | + |
| 28 | +<Callout type="info"> |
| 29 | + Running Voicebox in Docker instead of the desktop app? The API is on |
| 30 | + `http://127.0.0.1:17600` — set `VOICEBOX_BASE_URL` accordingly wherever it |
| 31 | + appears below. See [Docker](/overview/docker). |
| 32 | +</Callout> |
| 33 | + |
| 34 | +## MCP: agent-invoked voice tools |
| 35 | + |
| 36 | +Hermes speaks MCP natively, and Voicebox ships a built-in |
| 37 | +[MCP server](/overview/mcp-server). Voicebox is in Hermes's approved MCP |
| 38 | +catalog, so: |
| 39 | + |
| 40 | +```bash |
| 41 | +hermes mcp install voicebox |
| 42 | +``` |
| 43 | + |
| 44 | +(Or add the block manually to `~/.hermes/config.yaml`:) |
| 45 | + |
| 46 | +```yaml |
| 47 | +mcp_servers: |
| 48 | + voicebox: |
| 49 | + url: "http://127.0.0.1:17493/mcp" |
| 50 | + headers: |
| 51 | + X-Voicebox-Client-Id: "hermes" |
| 52 | +``` |
| 53 | +
|
| 54 | +Hermes discovers the tools — `voicebox.speak`, `voicebox.transcribe`, |
| 55 | +`voicebox.list_profiles`, `voicebox.list_captures` — and the agent can now |
| 56 | +*choose* to use them: "read me that summary in Morgan's voice" works |
| 57 | +immediately, and the [per-client binding](/overview/mcp-server#per-client-bindings) |
| 58 | +for `hermes` lets you pin its default voice from the Voicebox UI. |
| 59 | + |
| 60 | +MCP makes Voicebox a set of tools the agent may call. It does **not** |
| 61 | +reroute Hermes's own voice pipeline — spoken replies, voice bubbles, and |
| 62 | +incoming voice-message transcription still use whatever `tts.provider` / |
| 63 | +`stt.provider` are set to. That's the plugin's job. |
| 64 | + |
| 65 | +## Provider plugin: Hermes's own voice pipeline |
| 66 | + |
| 67 | +[`hermes-voicebox`](https://github.com/jamiepine/hermes-voicebox) registers |
| 68 | +Voicebox as a Hermes **TTS provider** and **STT provider** via Hermes's |
| 69 | +pluggable backend interfaces (see the Hermes developer guides for |
| 70 | +[TTS providers](https://hermes-agent.nousresearch.com/docs/developer-guide/tts-provider-plugin) |
| 71 | +and |
| 72 | +[transcription providers](https://hermes-agent.nousresearch.com/docs/developer-guide/transcription-provider-plugin)). |
| 73 | +Once selected, the providers service the *entire* voice pipeline: every |
| 74 | +spoken reply, every Telegram voice bubble, every incoming voice memo — plus |
| 75 | +a bundled skill that teaches the agent when speaking aloud is appropriate |
| 76 | +and to recall your dictated [Captures](/overview/captures) through MCP. |
| 77 | + |
| 78 | +<Steps> |
| 79 | + |
| 80 | +### Install the plugin |
| 81 | + |
| 82 | +Into the same Python environment Hermes runs in: |
| 83 | + |
| 84 | +```bash |
| 85 | +pip install git+https://github.com/jamiepine/hermes-voicebox |
| 86 | +``` |
| 87 | + |
| 88 | +No pip? Copy it in as a directory plugin instead: |
| 89 | + |
| 90 | +```bash |
| 91 | +git clone https://github.com/jamiepine/hermes-voicebox /tmp/hermes-voicebox |
| 92 | +cp -r /tmp/hermes-voicebox/hermes_voicebox ~/.hermes/plugins/voicebox |
| 93 | +hermes plugins enable voicebox |
| 94 | +``` |
| 95 | + |
| 96 | +### Select the providers |
| 97 | + |
| 98 | +In `~/.hermes/config.yaml`: |
| 99 | + |
| 100 | +```yaml |
| 101 | +tts: |
| 102 | + provider: voicebox |
| 103 | +
|
| 104 | +stt: |
| 105 | + provider: voicebox |
| 106 | +``` |
| 107 | + |
| 108 | +### Try it |
| 109 | + |
| 110 | +With the Voicebox app open, start `hermes chat` and ask it to say |
| 111 | +something out loud — or send your Hermes bot a voice message on Telegram |
| 112 | +and watch the transcript come back from your local Whisper. |
| 113 | + |
| 114 | +</Steps> |
| 115 | + |
| 116 | +## Behavior notes |
| 117 | + |
| 118 | +- **Voicebox must be running.** The desktop app only serves the API while |
| 119 | + it's open. Both providers implement availability as a live `/health` |
| 120 | + check, so Hermes's provider picker reflects reality. |
| 121 | +- **First generation is slower** while the TTS engine loads into memory; |
| 122 | + subsequent calls are fast. Same for the first transcription with a new |
| 123 | + Whisper size — Voicebox answers `202` while the model downloads, and the |
| 124 | + plugin surfaces a friendly "try again in a minute". |
| 125 | +- **Voice selection**: `tts.voice` in Hermes config (or the tool's `voice` |
| 126 | + argument) accepts a Voicebox profile **name or id**. With no voice set, |
| 127 | + the first profile is used. |
| 128 | +- **Engines**: pass a Voicebox engine id (`qwen`, `kokoro`, |
| 129 | + `chatterbox`, …) as the Hermes `model` to override the profile's |
| 130 | + default engine. |
| 131 | + |
| 132 | +## Next steps |
| 133 | + |
| 134 | +- [MCP Server](/overview/mcp-server) — the tool-call route, per-client |
| 135 | + bindings, and the speaking pill |
| 136 | +- [Creating Voice Profiles](/overview/creating-voice-profiles) — clone the |
| 137 | + voice Hermes will speak in |
| 138 | +- [Remote Mode](/overview/remote-mode) — reaching a Voicebox instance on |
| 139 | + another machine (read the security notes first: the API has no auth) |
0 commit comments