Skip to content

Commit 22e8cc0

Browse files
AbirAbbasclaude
andauthored
Token/cost usage tracking end-to-end + tray usage UI (#795)
* feat(sdk/python): capture and transport token/cost usage per execution Record token usage even when pricing fails (the OpenRouter gap: unknown model slugs made litellm.completion_cost return None and the tokens were discarded with the cost). Adds OpenRouter native cost accounting (usage.include), Anthropic-native usage-shape extraction incl. cache tokens, Claude Code harness token capture, and a per-execution contextvar-scoped CostTracker whose serialized summary is attached to the execution result envelope (sync 200 body and async status callback) under the "usage" key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(control-plane): persist execution usage and serve stats/timeseries API New execution_usage table (goose migration 034 for Postgres, GORM auto-migrate covers SQLite) populated by tolerant ingestion of the SDK's "usage" envelope key on both the sync-200 and async-callback paths. New GET /api/ui/v1/usage/stats endpoint with window filtering, by-model / by-provider / by-agent / by-harness aggregation, zero-filled bucket timeseries (?buckets=N) and per-model series (&series_by=model). Existing execution-details responses now carry cost/total_tokens, which the web UI already renders. Includes a cross-language golden test pinning the Python SDK's serialized payload against the Go parser. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(af-tray): usage glance UI, lifecycle status badge, vendored systray fork The tray's menu-bar item now shows the af badge with a lifecycle glyph (green dot running, rotating arc starting, gray ring stopped). The Usage submenu is a minimal glance surface: a 48-bucket token histogram (stacked per model), top-3 model rows and Claude 5h/7d subscription quota gauges (read-only, via the user's existing Claude Code Keychain credentials), all on a uniform leading-slot grid so every title aligns. Success row gains a live sparkline icon; response latency is humanized. Vendors fyne.io/systray v1.12.2 as third_party/systray with two additive patches (SetImage / SetStatusImage) lifting the stock 16x16 menu-image clamp — see third_party/systray/PATCHES.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(docker): copy third_party module replacements before go mod download The go.mod replace directive for the vendored systray fork points at control-plane/third_party/systray, which was not present in the build context when Dockerfiles copied only go.mod/go.sum for the module download cache layer, failing the control-plane image and functional test builds. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(usage): namespace the sync-200 usage envelope key so user payloads survive The sync result path reserved a top-level "usage" key in dict results and relied on the control plane stripping it back out — silently mutating any agent result that legitimately returns its own "usage" key (review finding on #795). Usage now travels under the reserved "__agentfield_usage__" envelope key (USAGE_ENVELOPE_KEY / usageEnvelopeKey); the control plane extracts and strips exactly that key and never touches user data. Regression tests pin that a user-owned "usage" key — top-level or nested — passes through byte-for-byte. The async status-callback path already carried usage as a typed sibling field of "result" and is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sdk/typescript): capture and transport token/cost usage per execution Port of the Python SDK's usage tracking to the TypeScript SDK, emitting the same cross-language wire contract: - CostTracker bound to the ExecutionContext (AsyncLocalStorage-isolated per execution; local agent.call() rolls child usage into the parent tracker). - LLM capture in AIClient generate paths (AI SDK v6 usage incl. cache token details) and across ToolCalling loop turns; OpenRouter requests opt into native cost accounting (usage: {include: true}) via a fetch wrapper, with cost read from the provider's raw usage. stream() usage is intentionally not captured — draining the usage promise would consume abandoned streams. - Claude Code harness runs record tokens + provider-reported cost. - Transport: "usage" field on all terminal async status reports; "__agentfield_usage__" reserved sibling key on object-shaped sync-200 results (user payloads, including a user-owned "usage" key, untouched). 44 new vitest tests; suite 730 green, tsc + tsup clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(sdk/go): capture and transport token/cost usage per execution Port of the Python SDK's usage tracking to the Go SDK, emitting the same cross-language wire contract: - CostTracker bound per execution via context (fresh tracker at every handler entrypoint: execute, reasoner sync/async, skill), read back to attach usage after the handler returns. - LLM capture at the Agent.AI / AIWithTools / AIStream chokepoints; ai.Usage gains cache read/creation tokens (OpenAI prompt_tokens_details and Anthropic-native shapes) and provider-reported cost. OpenRouter requests opt into native cost accounting (usage: {include: true}) at the request marshal chokepoint; tool-call loops capture per-turn usage. - Harness runs record tokens + cost: token fields threaded through harness.Metrics/Result and populated from already-parsed provider output (claude code, codex, opencode). - Transport: "usage" field on async terminal status payloads; "__agentfield_usage__" reserved sibling key on object-shaped sync-200 results (user payloads, including a user-owned "usage" key, untouched). - Fixes a pre-existing SSE decoder bug that dropped bytes returned alongside io.EOF — the read where a terminal usage-accounting chunk arrives. 28 new tests; go build/vet/test green, -race clean on the new paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(control-plane): pin Go and TypeScript SDK usage payloads as goldens Extends the cross-language golden contract test to all three SDKs: the verbatim Serialize()/serialize() outputs of the Go and TypeScript trackers (generated by running each SDK's serializer) parse to the same rows as the Python fixture. Row assertions are shared; only the priced entry's cost_source differs by design (litellm for Python, provider for Go/TS). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(control-plane): compare usage windows by epoch, not raw timestamps Found by live end-to-end verification of the Go/TS SDK usage transport: on a control plane running in a non-UTC timezone, /usage/stats returned zeros for every bounded window (1h/24h/7d/30d) while window=all showed the rows. GORM stamps execution_usage.created_at with time.Now() in server-local time, SQLite stores timestamps as text, and text comparison across mixed UTC offsets is lexicographic — so "created_at >= <UTC since>" silently excluded in-range rows (and ORDER BY created_at could pick the wrong oldest/latest row). CI runners are UTC, which is why the existing window tests never caught it. All window filters and oldest/latest orderings now go through the existing dialect-portable epoch expression (SQLite strftime / PostgreSQL EXTRACT). The new regression test pins the behavior with rows whose created_at carries a fixed -04:00 offset; it fails on the old code even on UTC hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent eb879b2 commit 22e8cc0

120 files changed

Lines changed: 16870 additions & 227 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ examples/go_agent_nodes/multi_version
2727
control-plane/agentfield-server
2828
control-plane/af
2929
control-plane/af-tray
30+
control-plane/cmd/af-tray/af-tray
3031
scripts/lambda-rie-fake/lambda-rie-fake
3132

3233
# Python

control-plane/cmd/af-tray/assets/icons/LICENSE.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,3 +14,12 @@ They are rendered to 32×32 PNG (16pt @2x) as black-on-transparent template
1414
images so macOS recolors them to match the menu (light/dark).
1515

1616
dot-green / dot-red / dot-gray are original status indicators.
17+
18+
chart-column is an original column-chart glyph (drawn in the same
19+
black-on-transparent template style as the Lucide line icons) used for the
20+
Usage section.
21+
22+
The tray's rendered images (the stacked chart, share bar, proportional model /
23+
quota bars, and the menu-bar sparkline widget) are pure graphics and contain no
24+
embedded-font text; the numbers next to them are native menu / menu-bar text.
25+
166 Bytes
Loading
Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
package main
2+
3+
// Text-free image renderer for the Usage submenu's 24h timeline: a compact bucket
4+
// histogram, stacked by model. It is pure graphics (image/png only, no fonts, no
5+
// systray/CGO), so it compiles and is unit-tested on Linux CI. The per-model /
6+
// rollup / quota rows are NOT rendered here — they are native menu-item text
7+
// titles set by the darwin tray, each paired (models, quota) with a compact bar
8+
// image from charts.go's slotBarPNG. The darwin tray turns the PNG below into a
9+
// full-width menu-item image via the vendored systray fork's SetImage.
10+
//
11+
// All geometry is expressed in OUTPUT pixels — the 2x-retina PNG the tray hands
12+
// to SetImage — and multiplied by the supersample factor `ss` for the internal
13+
// high-res buffer, which is then box-downsampled for crisp bar edges and gaps.
14+
15+
import (
16+
"image"
17+
"image/color"
18+
"math"
19+
)
20+
21+
// ---- Usage submenu geometry (points) ---------------------------------------
22+
//
23+
// The design system: EVERY row in the Usage submenu carries a leading image of
24+
// exactly the uniform SLOT size, so every native title starts at the same x. Rows
25+
// with a graphic (model / quota) draw a compact bar inside the slot; rows without
26+
// one (summary lines, section headers, rollups, footer) get a fully transparent
27+
// spacer of the same size. The 24h histogram is the one wider, text-free graphic
28+
// row; its left edge still aligns with the slot, and its first bucket lines up
29+
// with the compact bars below. These constants live here (cross-platform) so the
30+
// pure renderers, their tests, and the darwin tray all share one source of truth.
31+
const (
32+
// Uniform leading slot every submenu row carries.
33+
usageSlotWidthPt = 64
34+
usageSlotHeightPt = 12
35+
// Compact proportional bar drawn inside the slot (model / quota rows),
36+
// left-aligned and vertically centered.
37+
usageBarWidthPt = 56
38+
usageBarHeightPt = 8
39+
// 24h histogram: a full row-content-width (slot + title area), text-free row.
40+
usageChartWidthPt = 200
41+
usageChartHeightPt = 28
42+
)
43+
44+
// ---- Bucket histogram ------------------------------------------------------
45+
46+
// histogram tuning, in OUTPUT pixels (2x retina).
47+
const (
48+
histTopPadPx = 3 // headroom above the tallest bar so it doesn't touch the top
49+
histGapPx = 1 // gap between adjacent buckets
50+
histStubPx = 1 // baseline stub height for empty buckets
51+
histMinNonEmpty = 2 // shortest a nonzero bucket ever draws, so a tiny value shows
52+
)
53+
54+
// histStubColor is the neutral baseline stub for empty buckets — dim enough not
55+
// to compete with the accent bars, present enough that the timeline reads as
56+
// continuous rather than broken around a lone spike.
57+
var histStubColor = color.NRGBA{grayOther.R, grayOther.G, grayOther.B, 0x59}
58+
59+
// histogramChartPNG renders the usage timeline as a compact bucket histogram: one
60+
// thin vertical bar per bucket (with 1px gaps), each bar stacking its models
61+
// bottom-up in the given hues (layers[0] at the bottom). Empty buckets keep a 1px
62+
// baseline stub so the timeline reads as continuous, which makes a lone spike look
63+
// like an event on a timeline rather than a broken chart. When series_by_model is
64+
// absent the caller passes a single layer (single accent hue). layers are
65+
// per-bucket token counts, all the same length; colors[i] tints layers[i]. It
66+
// carries no text — the numbers live in the native menu titles around it.
67+
func histogramChartPNG(layers [][]float64, colors []color.NRGBA, wPx, hPx int) []byte {
68+
if wPx <= 0 || hPx <= 0 {
69+
return nil
70+
}
71+
ss := barSupersample
72+
s := float64(ss)
73+
W, H := wPx*ss, hPx*ss
74+
hi := image.NewNRGBA(image.Rect(0, 0, W, H))
75+
76+
// Plot region. The left edge is flush (x=0) so the first bucket lines up with
77+
// the compact model bars in the rows below; the baseline sits on the bottom
78+
// edge; only a little headroom is reserved at the top.
79+
plotTop := float64(histTopPadPx) * s
80+
plotBot := float64(H)
81+
if plotBot <= plotTop {
82+
plotTop = 0
83+
}
84+
plotH := plotBot - plotTop
85+
86+
// Bucket count and the per-bucket stacked total.
87+
n := 0
88+
for _, l := range layers {
89+
if len(l) > n {
90+
n = len(l)
91+
}
92+
}
93+
if n == 0 {
94+
return encodePNG(downsample(hi, wPx, hPx, ss))
95+
}
96+
valueAt := func(li, i int) float64 {
97+
if li < 0 || li >= len(layers) || i < 0 || i >= len(layers[li]) {
98+
return 0
99+
}
100+
v := layers[li][i]
101+
if v < 0 {
102+
v = 0
103+
}
104+
return v
105+
}
106+
total := func(i int) float64 {
107+
sum := 0.0
108+
for li := range layers {
109+
sum += valueAt(li, i)
110+
}
111+
return sum
112+
}
113+
maxTotal := 0.0
114+
for i := 0; i < n; i++ {
115+
if t := total(i); t > maxTotal {
116+
maxTotal = t
117+
}
118+
}
119+
120+
stubPx := float64(histStubPx) * s
121+
minNonEmpty := float64(histMinNonEmpty) * s
122+
gap := float64(histGapPx) * s
123+
bucketW := float64(W) / float64(n)
124+
125+
fillRect := func(x0, x1, y0, y1 float64, c color.NRGBA) {
126+
xi0, xi1 := int(math.Round(x0)), int(math.Round(x1))
127+
yi0, yi1 := int(math.Round(y0)), int(math.Round(y1))
128+
for y := yi0; y < yi1; y++ {
129+
if y < 0 || y >= H {
130+
continue
131+
}
132+
for x := xi0; x < xi1; x++ {
133+
if x < 0 || x >= W {
134+
continue
135+
}
136+
blendPixel(hi, x, y, c, 1)
137+
}
138+
}
139+
}
140+
141+
for i := 0; i < n; i++ {
142+
xL := float64(i) * bucketW
143+
xR := xL + bucketW - gap
144+
if xR <= xL {
145+
xR = xL + 1
146+
}
147+
148+
t := total(i)
149+
if t <= 0 || maxTotal <= 0 {
150+
// Empty bucket: a 1px neutral stub on the baseline.
151+
fillRect(xL, xR, plotBot-stubPx, plotBot, histStubColor)
152+
continue
153+
}
154+
155+
// Bar height for this bucket, floored so a tiny nonzero value still shows.
156+
barH := (t / maxTotal) * plotH
157+
if barH < minNonEmpty {
158+
barH = minNonEmpty
159+
}
160+
scale := barH / t // pixels per token for this bucket's stack
161+
162+
bottom := plotBot
163+
for li := range layers {
164+
v := valueAt(li, i)
165+
if v <= 0 {
166+
continue
167+
}
168+
segH := v * scale
169+
top := bottom - segH
170+
if top < plotTop {
171+
top = plotTop
172+
}
173+
col := grayOther
174+
if li < len(colors) {
175+
col = colors[li]
176+
}
177+
fillRect(xL, xR, top, bottom, col)
178+
bottom = top
179+
}
180+
}
181+
182+
return encodePNG(downsample(hi, wPx, hPx, ss))
183+
}

0 commit comments

Comments
 (0)