Skip to content

Commit d6b36e2

Browse files
authored
Merge pull request #54 from Floe-Labs/docs/floe-guard-voice-adapters
docs: document Pipecat + LiveKit voice adapters and TS-parity position
2 parents b8faa47 + f1cd9a4 commit d6b36e2

1 file changed

Lines changed: 127 additions & 2 deletions

File tree

README.md

Lines changed: 127 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,10 @@ process.
1515

1616
Works with [CrewAI](#crewai) · [LiteLLM](#litellm) · [LangChain](#langchain) ·
1717
[LangGraph](#langgraph) · [OpenAI](#openai) · [Anthropic](#anthropic) ·
18-
[Gemini](#google-gemini) · [Vercel AI SDK](#vercel-ai-sdk) — or any stack, via
19-
plain `check()` / `record()`.
18+
[Gemini](#google-gemini) · [Vercel AI SDK](#vercel-ai-sdk) — and voice pipelines
19+
via [Pipecat](#pipecat-voice) · [LiveKit](#livekit-voice) — or any stack, via
20+
plain `check()` / `record()`. See the [adapter matrix](#adapter-matrix) for what
21+
ships in Python vs TypeScript.
2022
The hard-stop is contract-based: gate each call through the guard — adapters do
2123
it for LLM calls; for paid tools, [`reserve_tool()` / `settle_tool()`](#tool-spend-under-the-same-ceiling)
2224
block *before* the call runs (`record_tool()` alone meters a call after the
@@ -552,6 +554,129 @@ The middleware `check()`s before each call (throwing `BudgetExceeded` to halt th
552554
run) and `record()`s priced usage after — same semantics as the Python guard. See
553555
[`js/README.md`](js/README.md).
554556

557+
## Voice adapters (STT → LLM → TTS)
558+
559+
A voice pipeline has no single call site to wrap: the LLM sits inside a running
560+
session and turns fire continuously for the life of a call. So instead of a
561+
function wrapper, these adapters enforce **per turn** — reserve before the LLM
562+
call (so a turn is blocked *before* its TTS/audio spend piles on top of a call
563+
that would already cross the ceiling), settle on the real usage the pipeline
564+
reports, and release a turn that ends without ever reporting usage (an
565+
interrupted turn) so the reservation never leaks against the ceiling. The token
566+
cost map is LLM-only, so STT/TTS spend is metered only when you supply per-unit
567+
prices. Both voice adapters are **Python-only** today.
568+
569+
### Pipecat (voice)
570+
571+
```bash
572+
pip install floe-guard[pipecat]
573+
```
574+
575+
Drop a `FloeBudgetGuardProcessor` into the pipeline directly after the LLM
576+
service. It reserves on each turn's `LLMFullResponseStartFrame` and settles from
577+
the `LLMUsageMetricsData` Pipecat emits — so the pipeline's `PipelineTask` must
578+
be created with `enable_metrics=True, enable_usage_metrics=True`.
579+
580+
```python
581+
from pipecat.pipeline.pipeline import Pipeline
582+
from pipecat.pipeline.task import PipelineTask, PipelineParams
583+
from floe_guard import BudgetGuard
584+
from floe_guard.integrations.pipecat import FloeBudgetGuardProcessor
585+
586+
guard = BudgetGuard(limit_usd=1.00)
587+
588+
pipeline = Pipeline([
589+
transport.input(),
590+
stt,
591+
context_aggregator.user(),
592+
llm,
593+
FloeBudgetGuardProcessor(guard, model="gpt-4o"), # meters AND hard-stops
594+
tts,
595+
transport.output(),
596+
context_aggregator.assistant(),
597+
])
598+
task = PipelineTask(
599+
pipeline,
600+
params=PipelineParams(enable_metrics=True, enable_usage_metrics=True),
601+
)
602+
```
603+
604+
> **Fragment**`transport`, `stt`, `llm`, `tts`, and `context_aggregator` are your existing Pipecat objects; this shows only where the guard sits in a pipeline you already have. For a complete, runnable demo (no API key, no network), see [`examples/voice_turn_budget.py`](examples/voice_turn_budget.py).
605+
606+
By default a blocked turn pushes a fatal `ErrorFrame` that terminates the
607+
pipeline — the hard-stop every other adapter gives you. Pass an
608+
`on_budget_exceeded` async callback to speak a graceful "wrapping up" line first
609+
instead of cutting the call dead.
610+
611+
### LiveKit (voice)
612+
613+
```bash
614+
pip install floe-guard[livekit]
615+
```
616+
617+
`LiveKitBudgetGuard.attach(session, agent)` wires the reserve-before /
618+
settle-after contract onto a LiveKit `AgentSession`: it reserves in the agent's
619+
`llm_node` and settles on the session's `metrics_collected` `LLMMetrics`.
620+
LiveKit's `LLMMetrics` doesn't report the served model, so cost settles against
621+
the `model` you pass here.
622+
623+
```python
624+
from livekit.agents import AgentSession
625+
from floe_guard import BudgetGuard, ManualPrice
626+
from floe_guard.integrations.livekit import LiveKitBudgetGuard
627+
628+
guard = BudgetGuard(
629+
limit_usd=1.00,
630+
price_overrides={"gemini-2.0-flash": ManualPrice(0.30e-6, 2.50e-6)},
631+
)
632+
budget = LiveKitBudgetGuard(guard, model="gemini-2.0-flash")
633+
634+
session = AgentSession(...)
635+
budget.attach(session, agent) # wire reserve / settle / release
636+
await session.start(agent=agent, room=ctx.room)
637+
```
638+
639+
> **Fragment**`session`, `agent`, and `ctx.room` come from your LiveKit agent entrypoint (`JobContext`); this shows only where the guard attaches.
640+
641+
Pass `stt_usd_per_second` / `tts_usd_per_1k_chars` to also meter STT/TTS spend
642+
(often a voice agent's larger bill) via `record_tool`, and an `on_budget_exceeded`
643+
async callback to speak a wrap-up line before a turn ends. See
644+
[`examples/voice_turn_budget.py`](examples/voice_turn_budget.py).
645+
646+
## Adapter matrix
647+
648+
| Adapter | Python | TypeScript |
649+
|---|---|---|
650+
| OpenAI || via [Vercel AI SDK](#vercel-ai-sdk) |
651+
| Anthropic || via [Vercel AI SDK](#vercel-ai-sdk) |
652+
| Google Gemini || via [Vercel AI SDK](#vercel-ai-sdk) |
653+
| LangChain |||
654+
| LangGraph |||
655+
| CrewAI |||
656+
| LiteLLM |||
657+
| Vercel AI SDK |||
658+
| **Pipecat** (voice) |||
659+
| **LiveKit** (voice) |||
660+
661+
The TypeScript package is a single Vercel AI SDK middleware that guards any model
662+
the AI SDK wraps (OpenAI, Anthropic, Gemini, …) — it is not a per-framework
663+
adapter set like the Python package.
664+
665+
### TypeScript voice parity
666+
667+
**This release ships no TypeScript voice adapter.** The JS package
668+
([`js/`](js/)) ships one surface — the Vercel AI SDK middleware — and it is
669+
**text-only**: it guards a wrapped `LanguageModel`, not a running STT → LLM → TTS
670+
session. A voice/telephony builder on a Node stack should either drive the LLM
671+
turn through the Vercel AI middleware and meter STT/TTS spend by hand via
672+
`recordTool`, or run the LLM leg through the **hosted [Floe proxy](#upgrade-to-hosted-floe)**
673+
(un-bypassable, cross-vendor) — or use the Python Pipecat / LiveKit adapters
674+
above, which enforce the whole turn. No native TypeScript Pipecat/LiveKit voice
675+
adapter ships in this release.
676+
677+
For wiring floe-guard into an existing voice pipeline, see the Floe docs:
678+
**[Add Floe to your existing pipeline](https://floe-labs.gitbook.io/docs/getting-started/integrate-existing-pipeline)**.
679+
555680
## Honest about what this is
556681

557682
floe-guard is a **local, estimate-based** guardrail. It prices tokens from a

0 commit comments

Comments
 (0)