You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After 10,000 additions, you have an exact count. No cumulative rounding errors.
49
49
50
+
## Prompt Caching
51
+
52
+
Anthropic and OpenAI support prompt caching, where repeated system prompts are stored and reused at a significant discount - typically 90% cheaper for cached tokens. FastroAI automatically tracks cache tokens and factors them into cost calculations.
53
+
54
+
When you use FastroAgent, cache tokens are tracked automatically:
55
+
56
+
```python
57
+
response =await agent.run("What's the weather?")
58
+
59
+
print(f"Input tokens: {response.input_tokens}")
60
+
print(f"Cached tokens: {response.cache_read_tokens}") # Tokens from cache (90% discount)
61
+
print(f"Cost: ${response.cost_dollars:.6f}") # Accurate cost with cache discount
62
+
```
63
+
64
+
If you're using a long system prompt that gets cached, subsequent requests to the same agent will show cache hits. The cost calculation automatically applies the discounted rate for cached tokens.
65
+
66
+
**Requirements for prompt caching:**
67
+
68
+
-**Anthropic:** Prompts >= 1024 tokens with cache_control markers
69
+
-**OpenAI:** Prompts >= 1024 tokens on gpt-4o models
70
+
50
71
## Direct Calculator Usage
51
72
52
73
Sometimes you need cost calculation without running an agent - estimating costs upfront, building dashboards, or custom tracking:
Use `cost_microcents` when you need to aggregate costs across many calls - integer math won't drift. Use `cost_dollars` when displaying to users.
112
117
118
+
When prompt caching is enabled (Anthropic, OpenAI with long prompts), `cache_read_tokens` shows how many tokens came from cache at a 90% discount. This is automatically factored into `cost_microcents`.
119
+
113
120
## Structured Output
114
121
115
122
Getting strings back means you have to parse them. For anything structured - extracting entities, classifying content, generating schemas - use Pydantic models instead:
0 commit comments