Skip to content

Commit 2f4b497

Browse files
authored
Merge pull request #6 from Floe-Labs/feat/vercel-ai-sdk
feat: Vercel AI SDK middleware (TypeScript)
2 parents 51ef6be + bf1ecd7 commit 2f4b497

15 files changed

Lines changed: 4151 additions & 3 deletions

README.md

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -133,10 +133,30 @@ llm.invoke("hello") # checks budget before the call, records spend af
133133
The handler checks the budget on LLM start (raising `BudgetExceeded` aborts the
134134
call before it runs) and records token usage on LLM end.
135135

136-
### Coming next
136+
### Vercel AI SDK
137137

138-
The Vercel AI SDK (TypeScript middleware) is next. Open an issue if you want one
139-
sooner.
138+
The Vercel AI SDK is TypeScript-only, so it ships as a separate npm package that
139+
lives in [`js/`](js/).
140+
141+
```bash
142+
npm i floe-guard ai@4 @ai-sdk/openai
143+
```
144+
145+
```ts
146+
import { wrapLanguageModel } from "ai";
147+
import { openai } from "@ai-sdk/openai";
148+
import { BudgetGuard, budgetGuardMiddleware } from "floe-guard";
149+
150+
const guard = new BudgetGuard(5.0); // your ceiling, in USD
151+
const model = wrapLanguageModel({
152+
model: openai("gpt-4o"),
153+
middleware: budgetGuardMiddleware(guard), // throws before crossing
154+
});
155+
```
156+
157+
The middleware `check()`s before each call (throwing `BudgetExceeded` to halt the
158+
run) and `record()`s priced usage after — same semantics as the Python guard. See
159+
[`js/README.md`](js/README.md).
140160

141161
## Honest about what this is
142162

js/.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules/
2+
dist/
3+
*.tsbuildinfo

js/README.md

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# floe-guard (Vercel AI SDK)
2+
3+
**A local budget guardrail for AI agents** — the TypeScript counterpart to the
4+
[Python `floe-guard`](../README.md). It hard-stops your agent *before its next LLM
5+
call* when it would cross a USD spend ceiling. No account, no signup, no network.
6+
Runs in your process.
7+
8+
```bash
9+
npm i floe-guard ai@4 @ai-sdk/openai
10+
```
11+
12+
```ts
13+
import { wrapLanguageModel } from "ai";
14+
import { openai } from "@ai-sdk/openai";
15+
import { BudgetGuard, budgetGuardMiddleware } from "floe-guard";
16+
17+
const guard = new BudgetGuard(5.0); // your ceiling, in USD
18+
19+
const model = wrapLanguageModel({
20+
model: openai("gpt-4o"),
21+
middleware: budgetGuardMiddleware(guard),
22+
});
23+
// generateText / streamText with `model` now stop at $5 — the call that would
24+
// cross the ceiling throws `BudgetExceeded` BEFORE it runs.
25+
```
26+
27+
The middleware sits in the call path: it `check()`s before `doGenerate` /
28+
`doStream` (throwing `BudgetExceeded` to halt the run) and `record()`s priced
29+
token usage after — for streaming it reads usage from the `finish` part.
30+
31+
## Pricing
32+
33+
Tokens are priced **offline** from a bundled
34+
[LiteLLM cost map](src/cost_map.json). A model that isn't in the map (and has no
35+
manual price) **fails closed**: `record` throws `UnpriceableModelError` rather
36+
than silently treating spend as free — *you can't cap spend you can't measure.*
37+
38+
```ts
39+
const guard = new BudgetGuard(5.0, {
40+
priceOverrides: {
41+
"my-self-hosted-model": { inputCostPerToken: 1e-6, outputCostPerToken: 2e-6 },
42+
},
43+
// or failClosed: false to warn-and-skip for models you accept un-metered.
44+
});
45+
```
46+
47+
## Verified against
48+
49+
`ai@4` (`LanguageModelV1Middleware` via `wrapLanguageModel` /
50+
`experimental_wrapLanguageModel`). Declared as a peer dependency.
51+
52+
## Development
53+
54+
```bash
55+
npm install
56+
npm run build
57+
npm test
58+
npm run typecheck
59+
```
60+
61+
## License
62+
63+
MIT — see [../LICENSE](../LICENSE).

0 commit comments

Comments
 (0)